Strategy analyzer is trolling me

[jamesonjameson]

Hi there,

I created a strategy with the strategy builder, with 16 user inputs. Now I am trying to optimize them with the forward optimization. As you know, it would have trillions of iterations and would take millions of years to optimize all user inputs together. So I decided to do it in three steps.

So the first step was, to find the best configuration for data series, the perfect choppiness index period, and the perfect limit for the choppiness index, at what a trade shouldn´t be taken anymore. All the other user inputs I just let at the default settings for the first step. That was no problem, I had a total net profit of 9000$ with the walk forward function on the first step.

But when I use those optimized values for the second optimization step and add three to five other values, I get completely different results. Shouldn´t it be at least as good as the results from the first step? What is the reason, that I get different results?

[NinjaTrader_Emily]

Hello jamesonjameson,

Thank you for your post.

Please clarify the following so I may better understand and accurately assist you:

• What version of NinjaTrader are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.?.?.?)
• What "Backtest type" do you have selected? Are you running tests as a multi-objective optimization as well as a walk forward optimization?
• Do you see results when running the same/similar test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?
• Who are you connected to? This is displayed in green in the lower-left corner of the Control Center window.
• Are you connected to your data feed provider when running this test?
• What instrument(s) (and expiry if applicable) have you selected?
• What Data Series Type have you selected? Example: Tick, Minute, Day
• What From and To date is selected?
• Is your strategy a multi-instrument or multi-time frame strategy?
• Do you receive an error on the screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

I look forward to your reply.

[jamesonjameson]

-version: 8.1.1.7 64-bit
-Backtest Type: Walk Forward Optimization
-On Sample MA Crossover I don´t have this problem, there are the same and logical results
-I am connected to the datafeed I bought from NinjaTrader website, and I am connected while running the analyzer
-Instrument is MGC 12-23
-Data series I also want to get optimized, I typed in 3;15;1 for Range-timeframe
-Date ist from 30.08.2022 to 19.08.2023. 177 days optimization period, 177 days test period
-It is a multi timeframe strategy, for some Indicators I want a 16 Range input. I checked it multiple times in the strategy builder and couldn´t find a mistake there
-there is no error, only ridiculous results

[NinjaTrader_Emily]

Hello jamesonjameson,

Thank you for that information.

Does your strategy have IsInstantiatedOnEachOptimizationIteration set to true or false?

https://ninjatrader.com/support/helpGuides/nt8/isinstantiatedoneachoptimizationiteration.htm

If this is set to false, does your strategy reset class-level variables? There are two samples on that help guide page; one with IsInstantiatedOnEachOptimizationIteration set to true, and another with IsInstantiatedOnEachOptimizationIteration set to false. Each example explains what could happen with class-level variables.

I look forward to your reply.

[jamesonjameson]

IsInstantiatedOnEachOptimizationIteration is set to true in the code.

[NinjaTrader_Emily]

Hello jamesonjameson,

Thank you for your reply.

At this point, it doesn't seem to be a specific setting/configuration that is causing the discrepancy you are seeing in results. What I suggest is to add print statements throughout your strategy, such as when a condition to submit an order is true, when an order is filled, what prices orders are filled at, etc. You could even consider using StreamWriter to send this information to a text file. Then, run both types of tests and compare the output from the prints. This will help to identify any differences between the two tests and explain what is causing the discrepancy in results. For more details about using Prints to debug your scripts:

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working#post791121

If you are getting any output that you don't understand, please feel free to include your output here and I'd be glad to take a look at it with you. Output from the NinjaScript Output window may be saved by right-clicking and selecting Save As. This allows you to save it to a text file that can be easily attached to your forum reply.

Please let me know if I may be of further assistance.