Strategy Analyzer Backtesting - Order Fill Resolution vs Tick Replay

[gaz0001]

Strategy Analyzer Backtesting - Order Fill Resolution vs Tick Replay

I ran the same strategy and same backtest using three different settings.

All of the results were different.
Unclear which is the most accurate here.

Standard Backtest
Tick Replay = Off
Order Fill resolution = Standard
Net Profit = 7882

Tick Replay Backtest
Tick Replay = On
Order Fill resolution = Standard
Net Profit = 7227

High Resolution Backtest
Tick Replay = Off
Order Fill resolution = High (Tick 1)
Net Profit = 6194

[NinjaTrader_Emily]

Hello gaz0001,

Thank you for your post.

Order Fill Resolution and Tick Replay both have different functions, and you are not able to combine high order fill resolution with tick replay. Ultimately, to get the most accurate results, it depends on how the strategy was developed and what it does. There is no one-size-fits-all answer. My colleague has broken down this information in detail in the following forum post:

• https://forum.ninjatrader.com/forum/...ive#post773377

In a backtest, order fills are determined based on historical OHLC values for each bar. The standard fill resolution will use the OHLC values for the series type selected. The high order fill resolution allows you to select a second, more granular data series to get additional OHLC data for order fill processing. High order fill resolution may not be used with Tick Replay or in multi-time frame scripts. A more granular series may be added programmatically instead; if this is a strategy you developed and this option interests you, let us know and we would be glad to provide you with additional resources and examples on how to do this. Historical fill processing is described on the following help guide page:

• https://ninjatrader.com/support/help...ical_fill_.htm

Tick Replay is used to build historical bars tick-by-tick as if they were built live. OnBarUpdate() and OnMarketData() will be called sequentially as if it the script were running live. This can be helpful for strategies that rely on the most recent price, volume, ask, or bid prices during the processing of a bar. Scripts must be developed specifically to take advantage of Tick Replay, though it can also be used to have OnBarUpdate() processed On Price Change or On Each Tick when running a backtest instead of the default behavior of On Bar Close when Tick Replay is disabled. More information may be found here:

• https://ninjatrader.com/support/help...ick_replay.htm

We also have an overview of discrepancies between real-time results and backtest results here:

• https://ninjatrader.com/support/help...ime_vs_bac.htm

Please let us know if we may be of further assistance.

[gaz0001]

Hello gaz0001,

Thank you for your post.

Order Fill Resolution and Tick Replay both have different functions, and you are not able to combine high order fill resolution with tick replay. Ultimately, to get the most accurate results, it depends on how the strategy was developed and what it does. There is no one-size-fits-all answer. My colleague has broken down this information in detail in the following forum post:

• https://forum.ninjatrader.com/forum/...ive#post773377

In a backtest, order fills are determined based on historical OHLC values for each bar. The standard fill resolution will use the OHLC values for the series type selected. The high order fill resolution allows you to select a second, more granular data series to get additional OHLC data for order fill processing. High order fill resolution may not be used with Tick Replay or in multi-time frame scripts. A more granular series may be added programmatically instead; if this is a strategy you developed and this option interests you, let us know and we would be glad to provide you with additional resources and examples on how to do this. Historical fill processing is described on the following help guide page:

• https://ninjatrader.com/support/help...ical_fill_.htm

Tick Replay is used to build historical bars tick-by-tick as if they were built live. OnBarUpdate() and OnMarketData() will be called sequentially as if it the script were running live. This can be helpful for strategies that rely on the most recent price, volume, ask, or bid prices during the processing of a bar. Scripts must be developed specifically to take advantage of Tick Replay, though it can also be used to have OnBarUpdate() processed On Price Change or On Each Tick when running a backtest instead of the default behavior of On Bar Close when Tick Replay is disabled. More information may be found here:

• https://ninjatrader.com/support/help...ick_replay.htm

We also have an overview of discrepancies between real-time results and backtest results here:

• https://ninjatrader.com/support/help...ime_vs_bac.htm

Please let us know if we may be of further assistance.

Thanks for the reply Emily.

I understand that standard mode basically builds the bar then executes the trade afterwards, and it doesnt really know if your stop and take profit are inside the bar, which is hit first. Hence it can be innacurate, but a quick and rough way of checking a strategy.


I see the Tick Replay as running through the Market Replay, in high speed mode. It builds everything in real time, and pulls all the data through. Its super accurate.

However, where i get a bit confused is the Order Fill Resolution. If i have this set to 1 Tick, then it will run the analsyer tick by tick and essentially build the bars tick by tick and consolidate them into the timeframe resolution for the strategy.

I am unsure how the Tick Replay vs Order Fill Resolution = 1 tick would not be exactly the same.

[NinjaTrader_Emily]

Hello gaz0001,

Thank you for your reply.

The statement "If i have this set to 1 Tick, then it will run the analsyer tick by tick and essentially build the bars tick by tick and consolidate them into the timeframe resolution for the strategy." is not quite correct; using the high order fill resolution set to 1 tick will send a request to your data provider for 1-tick data and the analyzer will use the OHLC information for the 1-tick bars to determine if/when orders are filled and their prices.

Tick Replay is different from this in the way that it processes the strategy's logic programmatically. If you enable Tick Replay, the strategy's logic will be processed tick-by-tick but this does not affect the order fill algorithm which would still determine order fills based on the OHLC prices of the series using standard fill resolution. High resolution may not be combined with Tick Replay. In other words, TickReplay allows OnBarUpdate() to update for every price change or tick when Calculate is set to OnEachTick or OnPriceChange. This does not, however, provide information for accurate intra-bar fills. When enabled, orders will still fill with end-of-bar information.

1-tick intra-bar granularity provides that intra-bar information for orders to fill intra-bar. What you could do to ultimately have both options put into place is have the strategy developed so that it programmatically adds a 1-tick data series and submits orders to that 1-tick series and also enable Tick Replay. This will likely produce the results most close to real-time trading, though depending on the strategy and your needs you might be able to get similar results from using High Order Fill Resolution as well.

I appreciate your time and patience. Please let us know if you have any additional questions or concerns.