There seems to be a bit of confusion for what is happening here.
Yes optimizations use a significant amount of memory and yes, the memory is not decommitted right away.
NinjaTrader 7 had a way of forcing garbage collection and NinjaTrader 8 just lets .NET handle it.
Peter, when you wrote in over email, you gave a us a video showing your test with the SampleMACrossover strategy where it used a large amount of tick data to build 200 volume bars. That will have a noticeable impact on memory which we do see in your video. I performed this test, and then I continued testing to see if the memory was in fact not getting decommitted. After completing your test, I downloaded the same amount of data for ES 06-20 and before I ran the same optimization backtest on that data, the memory was decommitted. This tells me that .NET garbage collection is working and there is not a memory leak.
Something else to consider with optimization backtests: By default NinjaTrader will create new instances of the strategy for each optimization iteration. A strategy can be programmed to use IsInstantiatedOnEachOptimizationIteration=false (class level variables will need to be reset in State.DataLoaded) and the optimizer will not create new instances of the strategy for each optimization iteration. The additional optimization iterations will increase the amount of memory utilized, and will increase the the time needed to complete the backtest.
AI Generate strategies do not generate code that is compliant with IsInstantiatedOnEachOptimizationIteration=false, so AI Generate runs will carry a large processing time in nature.
SampleMACrossover utilizes IsInstantiatedOnEachOptimizationIteration=false, but I thought this would be worth mentioning for others running optimization backtests.
IsInstantiatedOnEachOptimizationIteration - https://ninjatrader.com/support/help...niteration.htm
Peter, if there is a test similar to the video provided where we can observe that memory is not decommitting after a period of time, we can look into that.
Otherwise, what you would be looking for is to have NinjaTrader implement a way to force garbage collection. We do have a feature request tracking interest behind this with the ticket ID SFT-2111. Let me know if you would like us to add a vote on your behalf.
I look forward to assisting.
Comment