Market Replay vs Back Test results

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

We cannot confirm if the backtest results will be better or worse than the results of the strategy running realtime.

This would be dependent on how you programmed the logic in your strategy.

Please review this forum post in depth for information regarding comparing real-time, historical, and replay performance: https://forum.ninjatrader.com/forum/...mance?t=102504

[Malletto]

Ok thank you. So what exactly is the issue with the kagi bars when backtesting that makes the results different? Maybe if you tell me i can figure out how to fix it. Is the market not there when it’s executing? Or something else? Also, what other type of bar/chart is most similar to kagi and will also be accurate for the backtest. Thx!

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

To have the strategy result in the backtest more accurately reflect results of the strategy running on realtime data, you would need to add intrabar granularity to your strategy as detailed on post # 27.

Please refer to post # 27 for information about adding intrabar granularity to a stratetgy to improve the accuracy of backtest results.

Also, review the help guide documentation and reference sample linked on post # 27.

[Malletto]

Ok i will look into that. But my question is what exactly is creating the discrepancy in the backtest so that i can compare

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

The discrepancies are from the difference in how historical data is processed compared to real-time data.

When in Historical data only the Open, High, Low, and Close price are known and no intrabar data is available. When in Realtime data there is intrabar data available for the strategy to process.

Please review the help guide document on the differences on real-time vs backtest (historical).
https://ninjatrader.com/support/help...ime_vs_bac.htm

As stated in post # 27: "When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time."

Adding intrabar granularity to the strategy will help with this so the backtest results more accurately reflect the results of the strategy running in realtime.

Please review this forum post in depth for information regarding comparing real-time, historical, and replay performance: https://forum.ninjatrader.com/forum/...mance?t=102504

[Malletto]

So what does this affect? Where it gets executed? Or also the indicator that is being used. I use trend magic to help you out

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

As stated in post # 27: "When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Below is a link to the help guide on Calculate.
https://ninjatrader.com/support/help.../calculate.htm"

This means that your strategy logic would only process at the close of a bar and the only information available would be Open, High, Low, and Close prices of that bar. The script will not have the intra-bar information to accurately fill an order at the exact price and time.

Add a 1-tick series to the script, submit orders to that added 1-tick series, and enable Tick Replay when running the backtest to improve the backtest results so they more accurately reflect the strategy running in realtime.

[Malletto]

But i think my strategy already does calculate on a close? Can you help me confirm that please. Thank you for the help

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

If you want your strategy backtest results in the Strategy Analyzer to more accurately reflect real-time results you must add intrabar granularity to your script.

Adding intrabar granularity to your script will improve your backtest results so they are closer to real-time results.

In the strategy, add a 1-tick data series to the script, submit orders to that added 1-tick series, and enable Tick Replay when running the backtest.

[Malletto]

What about calculating it on price change or every tick?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

It would be up to you if you want your strategy to calculate OnBarUpdate() logic at the close of each bar (Calculate.OnBarClose), for each change in price (Calculate.OnPriceChange), or for each incoming tick (Calculate.OnEachTick).

The Calculate property is set in the State.SetDefaults section of a script.

Regardless of the Calculate mode you use, intrabar granularity should be added to the strategy to improve backtest results so the closer match real-time results.

[Malletto]

Ok but how is it calculated on the backtest. I want to run it the exact same on the live market. Is it on bar close for backtest? Or on price change?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Backtests in the Strategy Analyzer can only process logic at the close of a bar (Calculate.OnBarClose).

From the "Discrepancies Realtime vs Backtest" help guide: "During backtest, strategies can ONLY be processed at the close of each bar."

Discrepancies: Realtime vs Backtest: https://ninjatrader.com/support/help...ime_vs_bac.htm

The discrepancies are from the difference in how historical data is processed compared to how real-time data processes.

When in Historical data only the Open, High, Low, and Close price are known and no intrabar data is available. When in Realtime data there is intrabar data available for the strategy to process.

By default when backtesting no intra-bar granularity is added. An added 1-tick series is necessary to increase fill accuracy to within 1 tick.

Further, TickReplay is necessary for intra-bar indicator values to update tick-by-tick historically as they would in real-time.

Adding a 1-tick secondary data series to the script, submitting orders to that secondary 1-tick data series, and enabling Tick Replay when backtesting the strategy will reduce the discrepancies so backtest results match closer to realtime results.

Further, see this help guide document about historical fill processing: https://ninjatrader.com/support/help...ical_fill_.htm

[Malletto]

Ok so this is my question. If the backtest is calculated on bar close and the real-time/live strategy is calculated on bar close. Isn’t that the same thing?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

This would be the same Calculate mode of OnBarClose.

However, intrabar granularity must be added to the script to increase the accuracy of the backtest performance.

An added 1-tick series is necessary to increase fill accuracy to within 1 tick.

TickReplay is necessary for intra-bar indicator values to update tick-by-tick historically as they would in real-time.

You have a ticket open with the Support team on this topic and we have replied to your question on that ticket. Please direct any other inquiries you have to that ticket email instead of posting the same questions on this forum thread.