Why do live data bars differ from market replay bars?

[QuantKey_Bruce]

Much of this was revamped completely in March when NinjaTrader 8.1 came out and the websites were all simultaneously updated. Everything is gradually getting clarified and made consistent and clear. When we find things that are contradictory or confusing we're letting NinjaTrader know and they're fixing them or clarifying any misunderstandings but it's all pretty new as much of this does not work the way it did as recently as February.

[QuantKey_Bruce]

Bruce just said you get LIVE free SIM data for the 14 days. After that you must sign up. Which is understandable. As long as you get at least 14 days to trade free LIVE data on sim.

Thank you Bruce for your reply. Much appreciated.

Kind regards

Anytime. Yes, you get live futures data for 14 days for free regardless with a new sign up. There is no cost.

[imonlysleeping]

Anytime. Yes, you get live futures data for 14 days for free regardless with a new sign up. There is no cost.

Right, but the live data is not real. It's pretend data. Isn't that what you said bruce? Go back to the first page and read the posts there from bruce. Perhaps i misunderstood. And perhaps peter doesn't care if its real or fake when he does his 14 days of pain lol

[QuantKey_Bruce]

If you are on a live account and paying for market data I don't think there is any question it is real data. If you are on a sim account and your 14 days have expired then it depends whether you are paying for live market data or not. That is my understanding. The name Simulation is confusing and has resulted in many posts and threads.

[imonlysleeping]

If you are on a live account and paying for market data I don't think there is any question it is real data. If you are on a sim account and your 14 days have expired then it depends whether you are paying for live market data or not. That is my understanding. The name Simulation is confusing and has resulted in many posts and threads.

So then after i had already immediately told you i was on a funded live account, why the hell were you talking to me post after post about simulation unless what you were telling me also pertained to having a live account (but signed in to the sim side)?????

Note: the name "simulation" is only confusing on NT because they've made it so. it needn't be and isn't if you go elsewhere.

[QuantKey_Bruce]

If you have signed up and are paying for live market data (the $12 or whatever the fee is for your account type) you should be getting live market data. If you did not sign up for a market data plan under Settings -> Plans -> Market Data then you would not be even if you have a funded account. I should have asked you to clarify if you signed up for paid market data and this is getting confused with the question of whether you have a funded trading account which is not really the important point. Traders can have paid real market data even without a funded account and often do - for instance, some use NinjaTrader for charting (for which they want non-delayed live market data) and place their trades in another application.

[imonlysleeping]

If you have signed up and are paying for live market data (the $12 or whatever the fee is for your account type) you should be getting live market data. If you did not sign up for a market data plan under Settings -> Plans -> Market Data then you would not be even if you have a funded account. I should have asked you to clarify if you signed up for paid market data and this is getting confused with the question of whether you have a funded trading account which is not really the important point. Traders can have paid real market data even without a funded account and often do - for instance, some use NinjaTrader for charting (for which they want non-delayed live market data) and place their trades in another application.

See my post 3 and post 10, i think its pretty easy to tell i have a live data subscription through NT. who trades live with real money without live data? this is all truly so far beyond my wildest imagination lol. anyway, thanks for trying i guess.

so let me get this straight -- are you now saying that as long as you have a live data subscription, then the live and sim side will all be the same live data and that that data is real? this would of course have been my initial assumption, but again, NT has me currently questioning all standard logic. and besides, you told me early on in this thread that the sim side was all fake fills on fake data. total confusion.

to go a step further, since i'm pretty sure this IS what you're saying, i think that takes us right back to my first post and finding out if the historical data i am seeing is the same data as i received through my paid data subscription. because as of right now, the historical data / playback data is NOT the same as the data which came in on the live data feed. this is still my current primary concern and question.

[QuantKey_Bruce]

Well, NinjaTrader_BrandonH's post referenced above notwithstanding, that is how I understand it. I'm sure NinjaTrader Support will clarify this further. It's a question of (a) are you in the 14-day trial? and if not (b) did you sign up to pay for market data under Settings -> Plans -> Market Data. If either (a) or (b) are true, you should be getting real data whether or not you are getting simulated fills. That is my understanding unless and until corrected by NinjaTrader.

[marcus2300]

Are you using Tick-based charts? Or Minute charts?

minute charts.

[imonlysleeping]

Bruce just said you get LIVE free SIM data for the 14 days. After that you must sign up. Which is understandable. As long as you get at least 14 days to trade free LIVE data on sim.

Thank you Bruce for your reply. Much appreciated.

Kind regards

This is true of course on the surface. But it has yet to be 100% confirmed whether that free live data is real or make believe. Bruce has pointed out that he is "pretty sure" that as long as you are paying for a live data subscription, then the data you receive on live side and sim side are all the same real live data feed. This would, of course, be the natural assumption.

However, it isn't confirmed precisely which data you are getting on the sim side (or the live side for that matter) during the 14 day trial.

And it is mostly confirmed, according to bruce, that if you AREN'T paying for a live data subscription, then the sim side is fake data and fake fills.

Maybe we will hear from an NT person to clarify this, as i'm sure they have to do continually since it isn't laid out plain anywhere that's easily accessible to the average new user.

[NinjaTrader_BrandonH]

Hello imonlysleeping,

Thanks for your post.

So that I may accurately assist, please answer all of the following questions.

• Who exactly are you connected to in the Connections menu when you mention you are connected to 'live' data? (NinjaTrader Continuum, Kinetick, Simulated, Live, etc)
• Are you subscribed to a Market Data subscription in the NinjaTrader Client Dashboard?
• What are the exact dates and times for the bars you are comparing in the screenshot you shared?
• What version of NinjaTrader are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.0.?.?)

I have tested comparing a 500-Tick ES 06-23 chart when connected to a realtime data feed connection to a 500-Tick ES 06-23 chart using Market Replay data on the Playback connection and the OHLCV bar values when comparing the charts are matching. See the attached screenshots.

First, I connected to a realtime data feed connection and opened a 500-Tick ES 06-23 chart. Then, I opened a Data Box window and noted the OHLCV values in the Data Box window when hovering the mouse over bars on the chart. Next, I downloaded Market Replay data for the ES 06-23 instrument for the dates 04/27/2023 and 04/28/2023.

After downloading Market Replay data, I connected to the Playback connection and set Playback to use Market Replay. I compared the same exact bar OHLCV values on the chart that I noted when connected to a realtime data feed connection and see that they do in fact match up exactly. The attached screenshots demonstrate that the bar values do match between a realtime data feed connection and Playback with Market Replay.

Are you able to reproduce the behavior you are reporting when following the same steps listed above?

The Simulation data feed connection creates simulated data locally on your PC (not based on real market movements). This connection is a random internally generated market and has NO correlation to real market data. This is stated in the help guide pages linked below.

https://ninjatrader.com/support/helpGuides/nt8/index.html?connections_menu.htm#UnderstandingThePreBuiltConnections

https://ninjatrader.com/support/help...connection.htm

Playback utilizes the ability to download or record market data and replay it at another time. For Market Replay data, NinjaTrader stores level I and level II together in a single market replay file to ensure that level I and level II events are perfectly in sync per instrument.

Market replay files have the ability to record time stamps down the 100 nanosecond level. However please note that we use the time stamp provided by the market data providers when storing data. This means that you are limited to the granularity of the provider if the time stamp is natively provided.

See the help guide pages linked below.
Understanding How Playback Data Works: https://ninjatrader.com/support/help...ePlaybackWorks
How to enable the market replay recorder: https://ninjatrader.com/support/help...ReplayRecorder

[imonlysleeping]

Connection: "Live"
Market Data: subscribed with NT
Date/Time: Friday Apr 28th, 7:56 - 7:58am
NT version: 8.1.1.3

I am attempting your test now and will respond with results. Thank you

[imonlysleeping]

The Simulation data feed connection creates simulated data locally on your PC (not based on real market movements). This connection is a random internally generated market and has NO correlation to real market data. This is stated in the help guide pages linked below.

1. if not subscribed to live data, but after the 14 day trial, what data is the Sim connection showing on the chart and what data is it using for strategy backtests, forward testing, and playback?

2. if not subscribed to live data, but prior to the 14 day trial end, what data is the Sim connection showing on the chart and what data is it using for strategy backtests, forward testing, and playback?

3. If subscribed to NT data, in any case, what data is the Sim connection showing on the chart and what data is it using for strategy backtests, forward testing, and playback?

[QuantKey_Bruce]

We have since established by clarifying questions that if you are not subscribed to live data you get delayed data but it is not simulation data - it is the same as live data just ten minutes later. There appears to have been a misstatement or confusing statement by support in the other thread I cited earlier which resulted in my misunderstanding the policy and I did not catch it since I always subscribe to live data and I simply repeated what was said by support with a link to their statement much earlier in this thread to explain why I was saying that.

We now know through subsequent clarifications that prior to the 14-day trial ending, you always get correct and non-delayed market data whether or not you subscribe, and after the trial ends, if you subscribed (or subscribe at any point thereafter) you get the same correct market data without a delay. If you let the trial end without subscribing, then until you subscribe, you get 10-minute delayed data for most common contracts e.g. ES futures etc. This is true whether you log in as simulation or as live. Subscribe is defined as signing up to pay the market data fees using Settings -> Plan -> Market Data on your user account portal at account.ninjatrader.com. Logging in as Simulation or Live gets you the same data with or without subscribing - you get delayed or non-delayed data based on whether you are on trial or have signed up to pay.

The earlier statement by support that Simulation is like the Simulated Data Feed (which is different) was confusing the two things because the wording is so similar that the wording got minced by accident and this has since been clarified. I've made a suggestion also to try to update some of this wording to make it less likely for this type of confusion or misstatement to happen. This is all new as of early March 2023 (less than two months ago) so some of us are still figuring out exactly how it works but I think it's all been clarified now. I did not catch that support mis-stated how this works earlier because I always pay for the market data and so I took the statement at face value when it was stated that Simulation was like Simulated Data Feed (without paid market data) but later it became clear that that was a statement made in haste or with less than perfect wording and Simulated Data Feed is something entirely different than Simulation.

We need to be careful, and I will be careful myself, not to say things like "simulation data feed" because the word "simulation" should always be used to consistently refer to how the fills are obtained (vs. a live market), while "simulated" should always be used to consistently refer to synthetic market data. It's a nuanced difference, but the wording is so close that it's easy to make a mistake here in this area.

So, to summarize, we now know and have gotten it clarified that:

(1) If you are in your 14-day trial or are paying for market data, you get regular non-delayed data. If you are after your 14-day trial and are not paying for market data, you get 10-minute delayed data. Either way, it is the same data, coming from the same place, and unchanged.

(2) You only get simulated market data (randomized) if you choose Simulated Data Feed (which is different than signing into Simulation). So just don't connect to Simulated Data Feed and this will not happen.

(3) It has nothing to do with whether your account is funded, and nothing to do with whether you signed into Live or Simulation, and only to do with whether you are either (a) on 14-day trial, or (b) are paying for market data.

I think everyone will agree this is much better and more consistent than the earlier understanding and is a good thing.

[NinjaTrader_BrandonH]

Hello imonlysleeping,

Thanks for your notes.

QuantKey_Bruce is correct in stating "Prior to the 14-day trial ending, you always get correct and non-delayed market data whether or not you subscribe, and after the trial ends, if you subscribed (or subscribe at any point thereafter) you get the same correct market data without a delay. If you let the trial end without subscribing, then until you subscribe, you get 10-minute delayed data for most common contracts."

The Simuation data feed connection will provide delayed market data and the data will be delayed by 10 minutes.

The Simulated data feed connection will automatically use the last price from the last connection as the starting price for the instrument and the data provided will be randomized.

The Strategy Analyzer uses only historical data to run backtests, optimizations, walk forward optimizations, etc. You must have access to historical data to use the Strategy Analyzer as stated on the Strategy Analyzer help guide documentation linked below. Historical data could be downloaded from the NinjaTrader historical data servers by using the Tools > Historical Data window.

Backtest a strategy: https://ninjatrader.com/support/help...a_strategy.htm
Optimization: https://ninjatrader.com/support/help...a_strategy.htm
Walk Forward Optimization: https://ninjatrader.com/support/help...e_a_strate.htm

The Playback connection could use Market Replay data downloaded from the NinjaTrader historical data servers or Historical data. See post # 72 for information about Market Replay data and review the help guide links on that post.

Please see the available historical data from the servers below.

• Tick data: up to 1 year of historical data would be available. However, some instruments may go back further.
• Minute data: virtually never deleted, goes back to 2006 (Forex goes back to late 2008).
• Daily data: virtually never deleted, goes back to 2009. Forex data varies per instrument. Index data goes back to 2011.
• Market Replay data: deleted after 90 days.