How to ensure a Genetic Optimization has enough memory to run...

[JPPoulin]

Hello fine NinjaTrader gurus!

With the Strategy Optimizer set to 'Genetic Optimization', I'm trying to set the parameters so that a run can complete.

Even with the settings set to default, my strategy begins to use nearly all my memory (my main machine has 96GB RAM) and then eventually starts trashing the SSD hard disk and eventually stops progressing and eventually crashes NinjaTrader.

Thus far even with all this memory I only get to about 108 of 125 iterations.

A bit about my strategy: it runs at 1 sec increments (so requires tick-level data), uses about 6 months of previous (tick) history (on an instrument like AMD) and usually takes 10 seconds to run 6 month of history 1-sec data when I run it in the default optimizer. (I have set IncludeTradeHistoryInBacktest to false so it takes less memory)

Q1: NT8 frequently uses 57GB of RAM and keeps requesting more (which will trash the HD for ultra-slow virtual ram) Is this normal?
Q2: Is there a way to force NT8 to use less RAM (some setting or command-line parameter?)
Q3: Can you guys run the Genetic Optimizer on 6-month of 1-sec tick data?

Thanks!

[NinjaTrader_Jesse]

Hello JPPoulin,

Q1: NT8 frequently uses 57GB of RAM and keeps requesting more (which will trash the HD for ultra-slow virtual ram) Is this normal?

You generally would see much lower use unless the system has this much ram available and you are doing a task that is taking that much memory. The platform does not force garbage collection so if your script continues to do tasks which require new memory it will continue to increase as the test goes on.

Have you tested the SampleMACrossOver in the same optimization to check how much memory is being used with that? This would be a good way to check the overall status of your platform and confirm if this is related to some of your custom logic.

If you are using drawing objects that may be one areas where you can eliminate memory use, drawing objects need to be created even in an optimization and will cause high memory use with a lot of iterations.

Q2: Is there a way to force NT8 to use less RAM (some setting or command-line parameter?)

No, this is controlled by the PC along with what you are using in the platform. If you are doing actions which require more memory it will establish that if it is available.

Q3: Can you guys run the Genetic Optimizer on 6-month of 1-sec tick data?

Yes, I just tried this on the SampleMACrossOver and did not see a large increase in memory usage, it only increased a very small amount. I would be interested to know what the result is from testing the SampleMACrossOver in the optimization you are now testing. If you see that the results are good and memory did not increase we can further troubleshoot the custom script, otherwise that will let us know something else is happening which we can look at more closely.




I look forward to being of further assistance.

[JPPoulin]

Thanks Jesse for the prompt and useful answer. Wished other software platforms had guys like you!

These are actionable ideas and I'll be checking today. I do indeed draw labels on the chart and I didn't think to draw them conditionally as I didn't think NT8 would draw them when the Strategy is run for internal purposes (not shown in GUI)

Q1: What would you recommend as the 'best test' to determine whether to draw my GUI elements or not? (
Should I use 'IsInStrategyAnalyzer', 'Owner is Chart', or some of the 'Category' enumerations (Category.Optimize?))

Q2: Other than drawing on the charts and printing to the log are there other actions that can slow down NT8 or consume excessive memory?

Q3: I notice from constantly looking at Windows System Monitor that system RAM remains very high even after several minutes of NT8 being idle after an optimizer run. You mentioned that garbage collection is not performed. Do you recommend I call it manually in my Strategy script? (Such as OnStateChanged.Terminated)

I'll try a run with SampleMACrossOver with the same input data and the same rig and report back... Surely it will eat less of 57GB of ram!

Thanks again!

[NinjaTrader_Jesse]

Hello JPPoulin,

I do indeed draw labels on the chart and I didn't think to draw them conditionally as I didn't think NT8 would draw them when the Strategy is run for internal purposes (not shown in GUI)

Yes this a commonly overlooked item as you would only visualize them in a chart. The platform still needs to account for them because they could potentially be used in a custom way such as referencing an object to get its price. If you are not doing anything that requires the objects in code and they are just visual turning them off will help with some performance.

Q1: What would you recommend as the 'best test' to determine whether to draw my GUI elements or not? (
Should I use 'IsInStrategyAnalyzer', 'Owner is Chart', or some of the 'Category' enumerations (Category.Optimize?))

If you wanted to automate it you could use a property like IsInStrategyAnalyzer or the Owner as you have shown, alternatively you can implement a user input bool to toggle this manually.

Q2: Other than drawing on the charts and printing to the log are there other actions that can slow down NT8 or consume excessive memory?

Likely yes however this is not necessarily something we have a comprehensive list of. This will generally relate to how the code was formed and what is being used. The only item I can say that will cause a drop in performance would be excessive drawing objects being used in an optimization. You can certainly do other C# logic which uses a lot of memory or cpu use. You have to keep in mind that nothing is being destroyed during the tests so doing a lot of storage or creating a lot of objects could add up. This would mainly relate to more complex objects like custom classes, drawing objects or objects which have a lot of "extra" to them. Basic types like int or double will take very small amounts of memory so it is not likely relating to simple types in the script.

Q3: I notice from constantly looking at Windows System Monitor that system RAM remains very high even after several minutes of NT8 being idle after an optimizer run. You mentioned that garbage collection is not performed. Do you recommend I call it manually in my Strategy script? (Such as OnStateChanged.Terminated)

No, we do not recommend that. Garbage collection is strictly handled by the .net framework in this situation. Calling the collection manually could cause problems later down the line as you may collect something which is not yet supposed to be collected at this point.

I'll try a run with SampleMACrossOver with the same input data and the same rig and report back... Surely it will eat less of 57GB of ram!

Yes please let me know once you have, if you see minimal resources being used I would suggest to make a quick test with the other script and disable the drawing objects to see what that changes. You can continue this process with other parts of your script to isolate the memory use. Keep in mind when testing situations like this, your script doesn't have to work like you want it to or perform accurately we just need to reduce what is being done in code to find the general problem. You can keep reducing parts of the script and while this will likely break it that will highlight what logic is required to see the problem.


I look forward to being of further assistance.

[JPPoulin]

Hi Jesse, here are some results that might interest you:
- I've disabled drawing (labels on the chart) with IsInStrategyAnalyzer and 1) the default GO args now complete in about 15 minutes (would crash about 80% of the way before because out of RAM) and 2) A GO operation with 100 iterations now complete with about 30GB for NT8 instead of 57+GB before. Turning off drawing is a MUST
- With the same input data I've run Sample MA CrossOver and it is *much* faster (about 2 minutes versus 15 for m strategy) and consumes much less memory (about 10GB instead of ~30GB)

I'm now going through my code making sure my external COM objects are released (MS Access recordsets)... I'll also turn off my slave indicators (Not call 'AddChartIndicator()' on them if GUI not shown...

Finally the jobs complete! Hopefully soon I'll be able to start jobs that run most of the night!