Market Replay vs Back Test results

[Malletto]

Thank you for this. Is there a way to make the realtime trade like the backtest. Maybe remove intrabar info? Or somehow calculate the same as backtest. Thx.

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

No, there are no means available for changing how real-time data processes as it comes from the broker/exchange. You also cannot change that real-time order fills are based on market dynamics.

You would need to use the Playback connection with Market Replay data for testing your strategies so the test results more accurately reflect real-time results.

Or, you would need to add a 1-Tick series to your script, submit orders to the added 1-tick series, and enable Tick Replay when backtesting in the Strategy Analyzer to allow your strategy to process historical data tick-by-tick as it would real-time and improve fill prices.

Please reference the SampleIntrabarBacktest example and the following Help Guide links for more information about adding intrabar granularity.

SampleIntrabarBacktest 'Backtesting NinjaScript Strategies with an intrabar granularity' - https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

TickReplay — https://ninjatrader.com/support/help...ick_replay.htm

Developing for Tick Replay -
https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub= developing+for+tick+replay

[Malletto]

That didn’t answer my question. How can you make the real time strategy act and execute like the backtest. Is it a calculation thing, what is it?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Backtests are executed on historical bars which always process with Calculate.OnBarClose. If your strategy uses OnEachTick or OnPriceChange for any logic then it won't match the backtest because that will be a difference in settings. To get similar processing you would need to run the strategy Calculate.OnBarClose in realtime and then also use prints in the strategy to verify that your logic executed identically to the backtest.

Ultimately, how your strategy executes both realtime and historically (backtest) is dependent on the logic and settings you use in the strategy.

For example, the SampleMACrossover strategy that comes default with NinjaTrader executes the same historically and realtime because it uses Calculate.OnBarClose processing and has simple conditions that don't have any realtime requirements.

You would have to go through your strategy's logic and settings to see if any of your logic relies on real-time, like using IsFirstTickOfBar for example, or using OnEachTick/OnPriceChange calculation such as for indicators used in the strategy.

If you do then real-time results will not match the backtest results and you would need to further modify your strategy logic to work OnBarClose in realtime.

That said, in general backtesting is not a reflection of what the strategy absolutely will do in real-time. It is going to provide results based on a specific set of rules and processing in a certain way which can differ from real-time.

[Malletto]

Ok thank you. Can you just clarify how the backtest calculates? And then I’ll make my realtime strategy execute the same. This is with Kagi bars by the way. Thank you for being so helpful

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

The Strategy Analyzer uses Calculate.OnBarClose processing when backtesting a strategy.

This means that all calculations and all actions occur at the close of a bar.

This is noted on the "Discrepancies: Realtime vs Backtest" help guide page linked on post # 60.Please review that help guide page for more information on this.

[Malletto]

I thought it was OHLC?

[Malletto]

Also, why do you think my indicator is acting different on the backtest compared to real time? Its the trend magic so its easy to notice the color difference.

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

No, I think you may be confused between how the Strategy Analyzer processes data and what bar information is available in historical data.

The Strategy Analyzer uses Calculate.OnBarClose for processing logic in a strategy when backtesting the strategy.

When in Historical Data only the OHLC information of a bar is known since that is the only information available in Historical Data.

This is noted on the "Discrepancies: Realtime vs Backtest" help guide page linked on post # 60. I suggest reading through that help guide page for details on this topic.

As previously stated: "To get similar processing you would need to run the strategy Calculate.OnBarClose in realtime and then also use prints in the strategy to verify that your logic executed identically to the backtest.

You would have to go through your strategy's logic and settings to see if any of your logic relies on real-time, like using IsFirstTickOfBar for example, or using OnEachTick/OnPriceChange calculation such as for indicators used in the strategy.

If you do then real-time results will not match the backtest results and you would need to further modify your strategy logic to work OnBarClose in realtime."

The TrendMagic indicator you are using may require Calculate.OnPriceChange or Calculate.OnEachTick processing for logic in the indicator script. Since backtests use Calculate.OnBarClose it won't match the backtest because that will be a difference in settings.

You could contact the developer of the indicator directly to confirm if the indicator requires OnPriceChange or OnEachTick for processing logic.

[Malletto]

Thank you so much brandon. So to clarify, trend magic calculates on Calculate.OnPriceChange or Calculate.OnEachTick But i need to switch it to on bar close?

[Malletto]

Do you also know why the candles are different when comparing the backtest and real time. Some candles are larger, or just in a different spot. There definitely different

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Without seeing the Trend Magic script, I cannot say for certain if it uses OnPriceChange or OnEachTick processing. You would either have to view the script to see what it uses for calculations or contact the developer directly and ask them.

You could consider modifying the Trend Magic script to use Calculate.OnBarClose instead of Calculate.OnPriceChange or Calculate.OnEachTick.

Note that you should ensure that the logic in the script does not require OnPriceChange or OnEachTick processing, such as using IsFirstTickOfBar in the script which would require using OnPriceChange or OnEachTick.

Regarding the candles being different, you would need to make sure that the same exact data and data series settings are being used for both the chart window and the Strategy Analyzer backtest.

When opening a New > Chart window set to use the exact same data being tested over in the Strategy Analyzer, I see the bars match.

See this demonstratio: video.https://brandonh-ninjatrader.tinytak...Nl8yMjQ5Mzg3OA

[Malletto]

Ok thank you. Do you know where in the afript

[Malletto]

Didn’t mean to press post… do you know where in the script it’ll show that? Or is there a way to see how it calculates without going into the script

[Malletto]

So basically the issue is that the trend magic is calculating differently on the analyzer compared to the real time executions.