Strategy Analyzer not Running Efficiently

[thereyoflite1]

Hello,

I am using a dedicated virtual machine with 120 cores and 480gb+ of ram at 2.44ghz

Despite these statistics, my optimizations are still slow and only using (AT BEST) 40% of the cores. Obviously, this is 48 cores which is still 'good', but the calculations are still very slow and leaving much to be desired (the remaining 72 cores).

I have read other forum posts to try and run intensive strategies to use a greater number of cores, but even in this I have had no luck (1;200;1 MACrossover strategy with 12 months of 1 minute data). I have also attempted optimizing other strategies with several parameters at a wide range. Still, nothing over 40% CPU utilization. The sole purpose of using this VM is the fast optimization of strategies.

I have tried all the recommendations I've seen in other forum posts, but quite frankly I am stuck. How do I get NT8 to take advantage of the complete processing power to get optimizations done faster?

Thank you

[NinjaTrader_Jim]

Hello thereyoflite1,

Thanks for your post.

We note in the System Requirements page of the Help Guide that 64 cores would be a limit. This is because NinjaTrader is not a "processor group aware" application.

Publicly available resource for more information on processor groups can be found here - https://bitsum.com/general/the-64-co...s-and-windows/

48/64 cores utilized sounds like you are getting most of those resources. I imagine some bottlenecking that you see takes place depending on the specific strategy instances that are being optimized.

For example, if you modify SampleMACrossover strategy so...

• it adds additional inputs that do not add any function, just raises the number optimization iterations,
• always has Fast and Slow set at low numbers like 1 and 2,

we would have many optimization iterations performing the same busy backtest where we have numerous trades simulated because of SMA(1)/SMA(2) crossovers.

I imagine that sort of optimization would show more even distribution on the 64 cores available to NinjaTrader.

Please let us know if you have any additional questions.

[thereyoflite1]

Hello,

Thank you so much for your response, the resources you provided were informative.
I went for higher ghz at 64 cores, and have been able to get desired core usage.

Are there any plans to make NT into a processor group aware application?

[thereyoflite1]

Running an optimization with several parameters over three months I am seeing uneven distribution of work across cores again. At times it is using 100% of 1 core while other cores remain at 0. Is there anything in code that usually causes this? I do not want to stop the optimization to alter code after making this much progress but would like to know for future reference.

[NinjaTrader_Jim]

Hello thereyoflite1,

We are tracking interest in supporting processor groups with the ticket ID SFT-5067 and I have added a vote on your behalf. This is an internal number, but for anyone else wishing to have their interest tracked, please let our support staff know that you would like a vote added for this request.

Feature Request Disclaimer

We receive many requests and cannot reasonably implement all requested features or changes. Interest is tracked internally and if enough interest is tracked, it would be weighed against how feasible it would be to make those changes to consider implementing. As such we cannot offer an ETA or promise of fulfillment.

When new features are implemented, they will be listed in the Release Notes page of the Help Guide. The ID number will be different than the internal feature request tracking ID, but the description of the feature will let you know if that feature has been implemented.

Release Notes - https://ninjatrader.com/support/help...ease_notes.htm

As for the question on even CPU utilization, NinjaTrader will assign one optimization iteration per worker thread, so the amount of utilization seen in one core would be dependent on what is processing in a specific optimization iteration. Some iterations may be more CPU intensive than others. For example, a backtest of SampleMACrossover with Fast 1, Slow 2 would have numerous trades generated and the work involved with that test could be much higher than Fast: 15 Slow: 30, where there would be less trades processed.

I would expect a test like I described in post #2 where we can essentially set up the same backtest with each optimization iteration to have a more even distribution.

[bltdavid]

We are tracking interest in supporting processor groups with the ticket ID SFT-5067

+1

Please add my vote.

[afcmaff]

NinjaTrader_Jim Please add my vote to SFT-5067 - thanks!

[NinjaTrader_BrandonH]

Hello bltdavid and afcmaff,

Thanks for your notes.

I have added your votes for this feature request.

Let me know if I may assist further.

[mafon]

Please add my vote to SFT-5067.

[NinjaTrader_BrandonH]

Hello mafon,

Thanks for your notes.

I have added your vote to the feature request.

When new features are implemented, they will be listed in the Release Notes page of the Help Guide.

[TKtrader14]

Hello thereyoflite1,

We are tracking interest in supporting processor groups with the ticket ID SFT-5067 and I have added a vote on your behalf. This is an internal number, but for anyone else wishing to have their interest tracked, please let our support staff know that you would like a vote added for this request.

Feature Request Disclaimer

We receive many requests and cannot reasonably implement all requested features or changes. Interest is tracked internally and if enough interest is tracked, it would be weighed against how feasible it would be to make those changes to consider implementing. As such we cannot offer an ETA or promise of fulfillment.

When new features are implemented, they will be listed in the Release Notes page of the Help Guide. The ID number will be different than the internal feature request tracking ID, but the description of the feature will let you know if that feature has been implemented.

Release Notes - https://ninjatrader.com/support/help...ease_notes.htm

As for the question on even CPU utilization, NinjaTrader will assign one optimization iteration per worker thread, so the amount of utilization seen in one core would be dependent on what is processing in a specific optimization iteration. Some iterations may be more CPU intensive than others. For example, a backtest of SampleMACrossover with Fast 1, Slow 2 would have numerous trades generated and the work involved with that test could be much higher than Fast: 15 Slow: 30, where there would be less trades processed.

I would expect a test like I described in post Forum where we can essentially set up the same backtest with each optimization iteration to have a more even distribution.

Please add my vote also

[NinjaTrader_BrandonH]

Hello TKtrader14,

Thanks for your note.

I have added your vote to this feature request.

When new features are implemented, they will be listed in the Release Notes page of the Help Guide.

[binarytech]

Please add my vote also

Please add my vote . Thank you!

[NinjaTrader_BrandonH]

Hello binarytech,

Thanks for your notes.

I have added your vote to this feature request.