Live versus Replay

[shil111]

Hi, I developed a strategy that focuses on very low profit target (a few ticks at most). This is not really HFT, as I found it technically impossible to implement under the available hardware due to slippage. It is a little slower and calmer than that, taking decisions in a few seconds. In my extensive tests, my strategy showed consistent profitability when run on the replay (tick level). However, it seems to lose all its stability when run on live data... In 'OnMarketData,' I used only the 'LAST' event, in an attempt to minimize potential differences between live data and replay - but obviosly it did not meet my expectations... I would appreciate any insight or ideas on the source of the discrepancies between live and replay modes, the points where the performance difference can be bridged, and where it cannot. Any other relevant suggestions would also be appreciated. Thanks.

[NinjaTrader_ChelseaB]

Hello shil111,

You may find the video 'Comparing Real-Time Performance with Historical or Playback Data' on the Comparing Strategy Results support article very helpful.

https://support.ninjatrader.com/s/article/Developer-Guide-Comparing-Strategy-Results

[shil111]

Thanks for your response and for the video. I watched it with great interest. I now understand that differences in execution time are expected between tick replay and live streaming - which makes sense, as every user has their own hardware and connection attributes.

However, I am still unclear on one thing and would appreciate clarification: Does the replay contain the full, real, tick-level trading data from the original live feed it was recorded from? Or, for example, during periods of violent price movement, is there any resolution reduction or aggregation?

This specific detail is critical for accurately pinpointing the source of the problem and identifying possible solutions.

Thanks again.

[NinjaTrader_ChelseaB]

Hello shil111,

TickReplay uses historical tick data. This is recorded from a live feed by the connected data provider, however this is only last transactions. Historical tick data does not contain all ask and bid market updates. Further, the historical tick data can be aggregated by the data provider.

Market Replay data (in Playback) does contain the ask and bid market updates and is not aggregated.

[shil111]

Just to be clear on terminology: Isn't "TickReplay" and "Playback" the same thing? I thought there are two ways to simulate: backtesting at the (historic) bars level,and by the "playback" that contian all "Last" updates.

[NinjaTrader_ChelseaB]

Hello shil111,

No, these are not the same.

TickReplay is used only in State.Historical. This allows the Calculate property to update for each tick or price change with Calculate.OnEachTick and Calculate.OnPriceChange.

Below are links to the help guide.

https://ninjatrader.com/support/helpGuides/nt8/tick_replay.htm

https://developer.ninjatrader.com/docs/desktop/developing_for_tick_replay

Playback is a connection that you can connect to that will playback recordings (Market Replay) or playback tick data (Historical tick data) as real-time.

Below are links to the help guide.

https://ninjatrader.com/support/helpGuides/nt8/playback_connection.htm

https://ninjatrader.com/support/helpGuides/nt8/data_files.htm

Market Replay data is different than historical tick data. Market Replay is more of a full market recording, while historical tick data is the price and time of each last tick.

[shil111]

OK. Thanks.

[shil111]

Assuming the "market replay" records the live session fully and without any reduction or compression, this leaves the possible causes for the very large discrepancy in performance I see between live and recorded instances (of the same session) as:

1. Differences in timing – This is very reasonable. The playback cannot and should not simulate each user's unique hardware, connection speed, etc. No real problem here.
2. How well the playback simulates the limit orders queue – because If the simulator makes filling our orders easier then in reality, this is a game changer for the worse, and the effectiveness of the replay as a development tool becomes much more limited, and even questionable, at least for medium to high rate trading.

I will continue digging, and if I reach a conclusion, I will share it with the community.