Market Replay vs Back Test results

[NinjaTrader_BrandonH]

Hello Ymcapital and Malletto,

Thanks for your notes.

The issue here is really the use of kagi bars. The issue of backtests being different from real time would be more easily seen with exotic bars types like Point and Figure and Kagi.

See this help guide page: http://www.ninjatrader.com/support/h.../bar_types.htm

To backtest those more accurately we normally suggest either running in Playback using Market Replay or submitting orders to an added 1 tick series in backtesting, so a higher fidelity fill can be worked with - however, the 1 tick series would simply provide more accurate fill prices.

You'd get the most accurate to real time results using Playback.

[Malletto]

I was told that backtesting does open, high, low, close. That’s the issue.

[Ymcapital]

I believe there's an issue with the timestamps in the NT backtest with Kagi bars. The new Kagi bar doesn't print until there is a reversal and in the backtest NT fills you retroactively. Generally speaking, using a secondary time series such as tick data can solve issues seen with backtesting Ninzarenko bars or HA candles but because of what I see with the backtests, tick series data doesn't solve this issue.

Brandon, can you confirm if that's accurate?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

I have addressed your questions in the support ticket you have open with the NinjaTrader Support deparment.

Please refer to the email I sent replying to your question and direct any further inquiries to that email instead of posting on this forum thread.

[Malletto]

But what if the strategy is a tick strategy?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

You must enable Tick Replay when backtesting the strategy so that indicator values in the script can update tick-by-tick historically as they would in real-time.

See this help guide page about Tick Replay linked on post # 27 for details about how Tick Replay works.

[Malletto]

Ok. But what else is the issue if there calculating on the same thing- bar close?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

This would be the same Calculate mode of OnBarClose.

However, intrabar granularity must be added to the script to increase the accuracy of the backtest performance.

An added 1-tick series is necessary to increase fill accuracy to within 1 tick.

TickReplay is necessary for intra-bar indicator values to update tick-by-tick historically as they would in real-time.

You have a ticket open with the Support team on this topic and we have replied to your question on that ticket. Please direct any other inquiries you have to that ticket email instead of posting the same questions on this forum thread.

[Malletto]

Ok so this is my question. If the backtest is calculated on bar close and the real-time/live strategy is calculated on bar close. Isn’t that the same thing?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Backtests in the Strategy Analyzer can only process logic at the close of a bar (Calculate.OnBarClose).

From the "Discrepancies Realtime vs Backtest" help guide: "During backtest, strategies can ONLY be processed at the close of each bar."

Discrepancies: Realtime vs Backtest: https://ninjatrader.com/support/help...ime_vs_bac.htm

The discrepancies are from the difference in how historical data is processed compared to how real-time data processes.

When in Historical data only the Open, High, Low, and Close price are known and no intrabar data is available. When in Realtime data there is intrabar data available for the strategy to process.

By default when backtesting no intra-bar granularity is added. An added 1-tick series is necessary to increase fill accuracy to within 1 tick.

Further, TickReplay is necessary for intra-bar indicator values to update tick-by-tick historically as they would in real-time.

Adding a 1-tick secondary data series to the script, submitting orders to that secondary 1-tick data series, and enabling Tick Replay when backtesting the strategy will reduce the discrepancies so backtest results match closer to realtime results.

Further, see this help guide document about historical fill processing: https://ninjatrader.com/support/help...ical_fill_.htm

[Malletto]

Ok but how is it calculated on the backtest. I want to run it the exact same on the live market. Is it on bar close for backtest? Or on price change?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

It would be up to you if you want your strategy to calculate OnBarUpdate() logic at the close of each bar (Calculate.OnBarClose), for each change in price (Calculate.OnPriceChange), or for each incoming tick (Calculate.OnEachTick).

The Calculate property is set in the State.SetDefaults section of a script.

Regardless of the Calculate mode you use, intrabar granularity should be added to the strategy to improve backtest results so the closer match real-time results.

[Malletto]

What about calculating it on price change or every tick?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

If you want your strategy backtest results in the Strategy Analyzer to more accurately reflect real-time results you must add intrabar granularity to your script.

Adding intrabar granularity to your script will improve your backtest results so they are closer to real-time results.

In the strategy, add a 1-tick data series to the script, submit orders to that added 1-tick series, and enable Tick Replay when running the backtest.

[Malletto]

But i think my strategy already does calculate on a close? Can you help me confirm that please. Thank you for the help