Questions on Market Replay & Strategy Analyzer for Accurate Strategy Testing

[Caspersky_]

Hi Team,

I'm currently exploring the Market Replay and Strategy Analyzer features in NinjaTrader and have encountered some discrepancies between the two. I'd like to understand these tools better for accurate strategy testing. Here are my questions:

1. Strategy Testing Accuracy: What is the most reliable method to test a trading strategy? Is the Strategy Analyzer a dependable tool for this purpose? If so, what are its limitations, particularly when using volume-based indicators? Should I prefer Market Replay in such scenarios?
2. Market Replay Data: Is there a more efficient method to download Market Replay data, perhaps in larger quantities or at a faster rate than the traditional method?
3. Contract Switches in Testing: When testing a strategy over a period that includes several contract switches (roll overs), what is the best approach to ensure accuracy using the Strategy Analyzer? Should the time ranges be aligned with the specific contracts traded during those periods? Please elaborate for both the Strategy Analyzer and Market Replay scenarios.
4. Rollover and Data Policy: In the context of contract rollovers, would using the MergeNonBackAdjusted policy yield more accurate results?
5. Historical vs. Market Replay: Could you clarify the differences between Historical data and Market Replay in the context of strategy testing?
6. Setting Up a Simulation Account: My demo account has expired. How can I set up a new simulation account for live-time strategy testing on the charts? What is the most accurate method to simulate strategies in real-time conditions? (Obviously I do have a live account ...)

I appreciate your guidance on these matters, as it will significantly aid in refining my trading strategies.

Casper.

[NinjaTrader_BrandonH]

Hello Caspersky_,

Thanks for your post.

You will see the most accurate testing results by using Playback with Market Replay data.

Market Replay data can only be downloaded one day at a time due to the size of Market Replay files. There are no options available for downloading more than 1 day of Market Replay data at a time in Ninjatrader.

See this help guide page for information about Market Replay data vs Historical data:
https://ninjatrader.com/support/helpGuides/nt8/index.html?playback.htm#UnderstandingHowThePlaybac kWorks

Please review the help guide document on the differences on real-time vs backtest (historical).

https://ninjatrader.com/support/helpGuides/nt8/discrepancies_real-time_vs_bac.htm

A strategy running real-time (live brokerage account, live market simulation, Playback connection etc...) will produce different results than the performance results generated during a backtest.

When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Below is a link to the help guide on Calculate.
https://ninjatrader.com/support/help.../calculate.htm

To improve the accuracy of a backtest, you may use Tick Replay along with an added 1-tick series to have logic processed intra-bar and have orders filled intrabar.
Tick Replay would be used to have the logic process OnEachTick or OnPriceChange with historical data, but this does not allow for intra-bar order fills. You would need to add a single tick data series and submit orders to that single tick data series for a strategy that uses Tick Replay.

High Order Fill Resolution allows for intra-bar order fills with historical processing, but is not compatible with Tick Replay.

Please reference the SampleIntrabarBacktest example and the following Help Guide links for more information.

SampleIntrabarBacktest 'Backtesting NinjaScript Strategies with an intrabar granularity': https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

TickReplay: https://ninjatrader.com/support/help...ick_replay.htm

Developing for Tick Replay:
https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub= developing+for+tick+replay

Additional information may be found in this NinjaTrader Forum post:
https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance?t=102504

For information about backtesting in the Strategy Analyzer and Merge Policy, see this forum thread:

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/96311-backtesting-choosing-contract-month#post782986

You could enable the 'Multi-provider' option in the Tools > Options > General tab to have access to the Sim101 account that comes default with NinjaTrader. The Sim101 account could be used for testing purposes.

[Caspersky_]

Thanks Brandon H.
Can you also please answer 3.Contract Switches in Testing regarding the instrument rollovers ?

[NinjaTrader_BrandonH]

Hello Caspersky_,

Thanks for your notes.

For information about backtesting in the Strategy Analyzer and Merge Policy, see this forum thread:

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/96311-backtesting-choosing-contract-month#post782986

You could use the front contract month and use either the MergeBackAdjusted or MergeNonBackAdjusted merge policy.

With MergeBackAdjusted (default setting) the chart will be continuous, stitched together by expiry month from the rollover date of the expiry month to the rollover date of the previous expiry month, and adjusted by an offset to prevent gaps between contract months (which trade at different prices) for a smooth chart.

With MergeNonBackAdjusted the chart with be continuous (stitched together by expiry month from the rollover date of the expiry month to the rollover date of the previous expiry month) but will not be adjusted and there will be gaps in price from one contract to the next.

With DoNotMerge only the data from the selected contact is shown and previous contract months will not be loaded. Chart is not continuous.

Merge Policy: https://ninjatrader.com/support/help...rge_policy.htm

In Market Replay, you should set the chart to use the instrument expiry that you are testing data over.