Why the difference between Strategy Analyzer and Replay?

[timmbbo]

I'm sure I'm not the first guy to wonder this.

I'll find one or two good strategies through the analyzer that seemingly make money. Then, I'll play those same strategies through the same instrument and time period through replay, and the strategies are crap. I'd expect a little more consistency between the two.

I'm not sure how to interpret this? Does it mean the strategy is bad, or my testing methodology is bad? Thanks for your thoughts.

[NinjaTrader_Kate]

Hello timmbbo,

Thank you for your note.

You should expect that a strategy running real-time (live brokerage account, live market simulation, Market Replay etc...) will produce different results than the performance results generated during a backtest either in the Strategy Analyzer or over historical data on a chart. This difference may be more easily seen on certain Bars types (e.g. Point and Figure, Renko) than others due to their inherent nature in bar formation. These bar types rewrite values based on calculations which cannot be historically simulated.

During a backtest you can select conservative or liberal fill algorithms which will produce different results. Fills are determined based on 4 data points, OHLC of a bar since that is the only information that is known during a backtest and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick'.

Also, here is a link to the differences on real-time vs backtest (historical).
http://ninjatrader.com/support/helpG...ime_vs_bac.htm

Adding intra-bar granularity can help with this.

Intra-bar granularity adds a second data series such as a 1 tick series so that the strategy has finer granularity in the historical data in between the OHLC of the primary series. This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with.

In NinjaTrader 8, there have been two new enhancements so that programmers do not have to manually add this secondary series and code the script to for high accuracy fills (Order Fill Resolution) and for intra-bar actions (TickReplay).

Here is a link to our forums that goes into depth on using Order Fill Resolution and Tick Replay to ensure your backtests are as close to real time results as possible:

https://ninjatrader.com/support/foru...mance?t=102504

High Fill Order Resolution and TickReplay cannot be used together. If it is necessary to have both, it is still possible to add intra-bar granularity to a script in the code itself for order fill accuracy and use TickReplay to update indicators with Calculate set to OnPriceChange or OnEachTick historically.

Please let us know if we may be of further assistance to you.