CPU and DDR usage limitation in backtesting.

[Czarek]

Hello. Are the any way to limit the pc resources usage by NT8 during optimization process?

[NinjaTrader_Jesse]

Hello Czarek,

There are not any specific controls on how the platform manages resources in the strategy analyzer. The optimization engine creates many instances of the strategy at once when testing. If you are seeing high memory usage that may be caused by using drawing objects. That can also be caused by other more complex coding scenarios as well.

[QuantKey_Bruce]

Currently it creates a thread pool with a number of optimization evaluation threads based on how many virtual cores are available (taking into account CPUs, cores and hyperthreading) and utilizes near 100% of the resources in order to minimize the run time to completion. How are you wanting it to work? Are you wanting some sort of max CPU % across the board? Or to reserve some cores for instance? I think you would have to think through a bit more what it is you are asking them to do because right now they're attempting to do the optimization as quickly as possible and you're asking them not to, basically.

Also, regarding memory usage (DDR is a particular kind of memory) every .NET program essentially uses all available memory and then initiates garbage collection when necessary. So, memory usage potentially can grow without bound until the .NET Framework decides to initiate garbage collection. It does not generally make sense for NinjaTrader to initiate garbage collection on its own, because generally, in designing such applications, you want the Framework to make these decisions. What that means, in the end, is that NinjaTrader has little control over how much memory is used. Still, if the computer has idle time, the Framework will periodically initiate garbage collection and memory usage will remain steady.

During optimization, because the CPU is quite busy, the Framework may forestall garbage collection until memory is nearly all used. That is how the Framework is designed. That is not really a problem per se - that's the design - to not waste time collecting the garbage until it is necessary to do so - unless there is free CPU and it has nothing else to do. This design philosophy is to ensure that .NET applications run as fast as possible by not slowing down critical tasks to do garbage collection until they must. Later, when the task is done, if the CPU is idle, the garbage will get collected even if not all the memory has been used. Additionally, if at any point, the memory gets close to being all used, it will do GC even if it's busy with critical tasks because it needs to. That is the design.