Strategy Analyzer vs Historical?

[Skifree]

I've built a strategy that uses pattern recognition and optimization to optimize dynamic signals over time. I've done most of my testing using Printouts and strategy analyzer, which i expect to differ from realtime trading, but testing can prove out the behavior and ability to optimize. My strategy benefits from a build up over time, so starting it without a historical state is means there's a lot of waste and it takes a few weeks to optimize.

I had expected to be able to use historical as this warmup process, but i'm noticing major differences between Strat Analyzer and Historical. Provided the same bars (enter on 5min, exit on 5 second) for the same periods, enough loading should produce results of some level of comparability. In strategy analyzer, it can take time but it often makes it's way to a pattern i can deal with. With Historical trading, I'm unable to reproduce this.

Is there a major difference between the two? I'm using the same parameters, same series but the range in variations between the two is very different. I've now tried far more times on Historical, unable to produce the results that seem like they should be comparable

[NinjaTrader_Gaby]

Hello Skifree,

Thank you for your post.

The Strategy Analyzer runs on historical data.

When you say you are testing in "historical", do you mean running the strategy on a chart or from the Strategies tab of the Control Center and only looking at the historical performance (before the script starts processing in realtime)?

I look forward to your response.

[Skifree]

Yes, running from the strategies tab and checking output or historical results

[NinjaTrader_Gaby]

Hello Skifree,

What Calculate setting is your strategy using?

Are you using TickReplay in the Analyzer, and if so are you also using Tick Replay in the Data Series the strategy is applied to?

I would suggest debugging using prints to determine where the discrepancy is coming from.

https://support.ninjatrader.com/s/article/Developer-Guide-Debugging-using-Print-and-TraceOrders

It may be beneficial to verify the data is the issue, and the issue is not caused something else, by printing the bar data or writing the bar data to a text file so that this can be compared.

In addition to prints for each condition, on each bar close, write the bar Time[0], Open[0], High[0], Low[0], and Close[0] to the text file.

The support article below provides an example script with sample code of writing the bar information to file that you can copy and paste into your script.

https://support.ninjatrader.com/s/article/Developer-Guide-Comparing-Strategy-Results

With the two text files, this will let us verify that the data is starting on the same bar and will let us compare to see if the data has changed.