What is the difference between Market Replay and live Data

[AdeptistJune]

I ran my strategy for the first 30 minutes of trading this morning. I took 17 trades and 900 bucks in winnings. I then switch to market replay and ran the strategy. It took 11 trades and 400 bucks in winnings. I thought market replay replay the live market in exact sequence? Any help please. Thanks

[NinjaTrader_Jim]

Hello AdeptistJune,

Thanks for your question.

Market Replay is a recording of live data as it is seen. While it is a complete recording of data in exact sequence, there could be some slight differences in testing when compared to live. Depending on how the strategy logic is sensitive to these differences, we could see different results.

• There is no order submission latency in Playback, while orders will need time to travel to the broker and exchange in a live scenario. This can result in slippage, which can change the PnL of trades, and if that is relevant to the strategy logic, it could control the number of trades taken.
• The origin of your regular realtime data feed may have slight differences than Playback. Market Replay data on our servers are recorded from CQG data.

Another important topic to consider is how much data was loaded, when exactly you enabled the strategy and the settings used. For example, if you connect to Playback and enable the strategy, but you have not enabled it at the same time the strategy was enabled live, it would not be an apples to apples test because the data on the chart before we hit the Enable tick box would be processed as historical data instead of realtime.

Discrepancies between Realtime and Backtest - https://ninjatrader.com/support/help...ime_vs_bac.htm

Keeping in mind the items above, if you perform tests with the Sim101 account on realtime data, and then enable the strategy on the same Market Replay data (enabling at the same time the strategy was enabled live,) we should expect near-same results with the exception of some variance in fill prices since Sim101 adds some random delays when switching between order states to simulate slippage.

Let us know if you have any additional questions.

[AdeptistJune]

Thanks Jim. That was an excellent explanation!

[AdeptistJune]

Hello,

I did every single thing to get the trades to match. My market replay trades are far different from my live trades. Is because I'm a different data feed than CQG?

[NinjaTrader_Jim]

Hello AdeptistJune,

It is possible the slight differences in data create differences, but if you have vastly different results, there is probably more at play.

To start, I would ensure the staring bars on the chart are the same between your Sim101 test, and then your on your Market Replay test. This should be set up when the Playback Controller is set to the same time you enabled the strategy, before the Play button is pressed. If this data appears the same, we should be able to enable the strategy at this same time with the same settings, and expect the historical results to match.

If there are differences in this part, it may be due to differences in data. I would then suggest using debug prints and checking the price levels, as well as to specifically look into how your conditions evaluate when trades are expected. These prints would tell you exactly why the the strategy did or did not take trades when it did.

(It may also be worth considering to implement a StreamWriter to output custom messages to a text file so you may more easily compare your Sim101 and Market Replay tests. These would serve a similar purpose as debug prints, but would be written to a text file instead of the output window.)

For the historical processing part, we should consider: same code + same data + same settings + same enablement time = same results. In order for the realtime results to be apples to apples, the historical part must match. If the results do not match, debugging prints/logs specifically comparing the data and how the strategy logic evaluates will point out the differences.

After the historical part is aligned when we start the strategy at the same time, and we then press Play, the realtime results should then align more closely. There will still be expected differences in fill prices and delays between switching order states, but your debugging output can still confirm strategy logic is becoming true on the same bars, or why it is not becoming true on those bars.