Announcement
Collapse
No announcement yet.
Partner 728x90
Collapse
NinjaTrader
Why do live data bars differ from market replay bars?
Collapse
X
-
Originally posted by NinjaTrader_BrandonH View Post
- Likes 1
-
Originally posted by NinjaTrader_BrandonH View PostThe Simuation data feed connection will provide delayed market data and the data will be delayed by 10 minutes.
The Simulation data feed connection will automatically use the last price from the last connection as the starting price for the instrument and the data provided will be randomized.
It is delayed and randomized?
What the heck does that even mean?
EDIT: I think Bruce caught one of several instances where NT
Support is confused by their own product. The root problem
is this: Somebody high in NT Mgmt decided it was OK to reuse
three same/similar words, "Sim, Simulated, and Simulation", which
already have definitions in prior releases, and expand one of them
with a brand new meaning specific to 8.1 -- and now, even NT
support can't keep all these meanings straight. Am I right?
Last edited by bltdavid; 05-03-2023, 08:23 AM.
Comment
-
bltdavid He has made a typo... the Simulated Data Feed connection is randomized market data. The Simulation connection has simulated fills but the same market data as the Live connection. It's easy to get these mixed up. That is exactly why I posted this suggestion thread to try to get this wording cleaned up properly.Last edited by QuantKey_Bruce; 05-03-2023, 08:18 AM.
- Likes 1
Comment
-
Hello QuantKey_Bruce,
Thanks for pointing that out.
Yes, this was a typo and I have corrected it in my previous post.
I had mean to say:
The Simuation data feed connection will provide delayed market data and the data will be delayed by 10 minutes.
The Simulated data feed connection will automatically use the last price from the last connection as the starting price for the instrument and the data provided will be randomized.Brandon H.NinjaTrader Customer Service
Comment
-
Yeah, I see that.
But when typos are so damn easy ... there's a problem here.
(See the EDIT of my previous post.)
I think the depth and breath of the stupidity of recent NT
decisions has just been widened ... 8.1 is reusing the word
'Simulation' in their sign-on process . that is, this word has
gained another meaning/usage in the 8.1 world ... and now
that term is too overloaded and causing confusing.
A realistic looking 'typo' like this, esp by NT support, can be
extremely difficult if not impossible to disentangle, esp for new
people, but also for experienced users -- it's a pity that users
have to know these very similar terms cold to correct NT
support. Good catch, Bruce.
Comment
-
Originally posted by bltdavid View PostA realistic looking 'typo' like this, esp by NT support, can be
extremely difficult if not impossible to disentangle, esp for new
people, but also for experienced users -- it's a pity that users
have to know these very similar terms cold to correct NT
support. Good catch, Bruce.Last edited by QuantKey_Bruce; 05-03-2023, 09:04 AM.
- Likes 1
Comment
-
Originally posted by NinjaTrader_BrandonH View PostThe Simulation data feed connection will provide delayed market data and the data will be delayed by 10 minutes.
The Simulated data feed connection will automatically use the last price from the last connection as the starting price for the instrument and the data provided will be randomized.
How in the heck is anyone (new or experienced)
going to realize these are two separate things?
Typo, my butt.
There is a larger problem at play here:
These two terms are too similar.
There is too much room for confusion.
(Obviously, Bruce, I agree with you on this)
The new 8.1 'Simulation data feed' concept is too similar to
the already existing 'Simulated Data Feed' feature.
In 8.1, the pre-built connection 'Simulated Data Feed' should
have been renamed to something like 'Fantasy Data Feed'
or 'Fake Data Feed' or 'Test Data Feed' or 'Synthetic Data
Feed' or 'Artificial Data Feed' ...
So, why didn't the existing one get renamed? NT Mgmt showed
arrogance on changing other things, so perhaps unawareness
of the confusion they're about to create, so it never occurred
to anyone to rename the old 'Simulated Data Feed' to make
way for the new 8.1 concept?
This would have avoided a lot of confusion by a lot of people,
even NT support people.
Comment
-
bltdavid Yes, we've been hashing out this same thing since March at https://forum.ninjatrader.com/forum/...n-vs-simulated and a few other threads. They're too similar. They need to be changed. I agree that changing Simulated Data Feed to Synthetic Data Feed is a decent way to resolve it.
- Likes 1
Comment
-
Hello bltdavid,
Thanks for your note.
As stated by Ryan on the forum post QuantKey_Bruce linked to and seen linked below, this has been reported to our product team and we are tracking customer feedback on changing the name of either the Simulation or Simulated data feed connections.
https://forum.ninjatrader.com/forum/...95#post1242895
I have added your notes on the internal ticket that we are tracking feedback on this topic for so the product team is aware of it and will take it into consideration. The ticket number is NT-3733.Last edited by NinjaTrader_BrandonH; 05-03-2023, 09:40 AM.Brandon H.NinjaTrader Customer Service
- Likes 1
Comment
-
Originally posted by QuantKey_Bruce View Postbltdavid Yes, we've been hashing out this same thing since March at https://forum.ninjatrader.com/forum/...n-vs-simulated and a few other threads. They're too similar. They need to be changed. I agree that changing Simulated Data Feed to Synthetic Data Feed is a decent way to resolve it.
As time has passed, it is clear NT mgmt doesn't care about
the user's ability to bypass the mandatory password requirement.
Frankly, the thought of upgrading really depresses me.
Thus, I'm not keeping up to date as I normally would.
Personally, I'm going to stay with pre '8.1' releases forever.
Sigh ... and it depresses me that 'forever' may not be possible.
I've added my feedback to your linked thread.
Thanks
Last edited by bltdavid; 05-03-2023, 09:39 AM.
- Likes 1
Comment
-
Originally posted by NinjaTrader_BrandonH View PostI have tested comparing a 500-Tick ES 06-23 chart when connected to a realtime data feed connection to a 500-Tick ES 06-23 chart using Market Replay data on the Playback connection and the OHLCV bar values when comparing the charts are matching. See the attached screenshots.
First, I connected to a realtime data feed connection and opened a 500-Tick ES 06-23 chart. Then, I opened a Data Box window and noted the OHLCV values in the Data Box window when hovering the mouse over bars on the chart. Next, I downloaded Market Replay data for the ES 06-23 instrument for the dates 04/27/2023 and 04/28/2023.
After downloading Market Replay data, I connected to the Playback connection and set Playback to use Market Replay. I compared the same exact bar OHLCV values on the chart that I noted when connected to a realtime data feed connection and see that they do in fact match up exactly. The attached screenshots demonstrate that the bar values do match between a realtime data feed connection and Playback with Market Replay.
Are you able to reproduce the behavior you are reporting when following the same steps listed above?
After following your test exactly as described above, i can conclude that:
The issue of OHLCV inconsistency between live data and market replay data is real and appears to be found all throughout any date/time ranges i choose.
My settings once again:
Connection: "Live"
Chart: ES Jun 23, 500 tick
Market Data: subscribed with NT
Date/Time: see screenshots below
NT version: 8.1.1.3
Here is just one screenshot comparison chosen randomly out of the MANY instances i'm seeing.
PLEASE NOTE:
Usually the disparity is just one tick off.
Volume and time are also both off.
The bar immediately following the labeled bar happens to also be a problem bar.
Last edited by imonlysleeping; 05-03-2023, 10:48 PM.
- Likes 1
Comment
-
Hello imonlysleeping,
Thanks for your notes.
The data pump can mean that ticks could be off by up to one second. This could be seen in the video shared about comparing real-time performance with historical or market replay data on the forum thread linked below.
https://forum.ninjatrader.com/forum/...nce#post100192
You can record market replay data from real-time data by enabling the 'Enable market recording for playback' option in the Tools > Options > Market data tab to make the market replay results more similar to the realtime data.
Record market replay data from real-time data: https://ninjatrader.com/support/help...ReplayRecorder
How to record live market data: https://ninjatrader.com/support/help...LiveMarketData
That said, I have attached a demonstration video here showing the market replay data matches or is very close to matching when comparing market replay data to realtime data: https://brandonh-ninjatrader.tinytak...M18yMTM5NzM0MQ
In the video linked above, we can see that the realtime bar at 09:39:44 has a Close price of 4075.50 and a volume of 647. When comparing this bar using Market Replay data on Playback, we can see that the 09:39:44 bar Close price and Volume match the bar when we were connected to realtime data.
We can also see the bar at 09:41:22 when connected to realtime data has a Close price of 4076.25 and a Volume of 796. When comparing this bar with Market Replay data on Playback, we see that the 09:41:22 bar Close price and Volume match. We then compare a third bar at 09:43:17 and see the Close price and Volume match again when comparing realtime data to Market Replay data.
Brandon H.NinjaTrader Customer Service
Comment
-
Originally posted by NinjaTrader_BrandonH View PostHello imonlysleeping,
Thanks for your notes.
The data pump can mean that ticks could be off by up to one second. This could be seen in the video shared about comparing real-time performance with historical or market replay data on the forum thread linked below.
https://forum.ninjatrader.com/forum/...nce#post100192
that thread you linked to represents all that i hate about ninjatrader. yuck. the longest craziest explanations for such extreme technical mayhem that it just makes you want to find another platform. its SUCH A MESS and so difficult to sift through all of that madness. i could spend all day asking questions about the unclarified, ambiguous statements in that thread and i'd probably end up more confused than when i started, as has been the usual way it goes in this forum.
do you think that if long elaborate explanations are necessary to answer simple questions, maybeeee the platform needs work?
OBJECTIVELY, its way way way too much for an average user or programmer to have to sift through in order to get results that are meaningful and valid. things should just work. and if they don't work simply, then they shouldn't be found anywhere on the platform. this is common sense.
did NT ever think to just make things good? remove all the options and the trash, and just make it good and accurate so i don't have to worry about or question every single little thing. right? does anyone see this madness and agree with me? i've never spent this much time on a forum and had to go around and around in circles like this and only ended up worse off than when i started, especially in terms of general frustration/confusion.
NT has a lot of potential, but there is a major oversight happening where common sense has been replaced with technical overload, when it just needn't be that way.
high user case functionality and design needn't be a complicated mess. the best things in the technical realm are simple and elegant and don't need explanation because they have a high level of intuitiveness built in for the user. NT lacks this in more ways than one. convince me otherwise.Last edited by imonlysleeping; 05-04-2023, 11:20 AM.
Comment
-
Hello imonlysleeping,
Thanks for your notes.
When using market replay, the NinjaTrader core market data updates occur at the granularity provided by the market data provider. However, the NinjaTrader user interface only visually updates in 1-second intervals for performance optimizations. Even though the NinjaTrader UI's are only visually updating at 1-second intervals, orders, indicators, and strategies will calculate just as they were running in real-time.
This is noted in the "Understanding how the Playback Works" section of the help guide linked below.
https://ninjatrader.com/support/help...ePlaybackWorks
The concerns you mentioned have been noted and will be taken into consideration. I submitted a feature request to the Development team for you regarding adding the ability to make the backtest engine automatically implement 1-tick intra-bar granularity (at the cost of performance). This request is being tracked under the number SFT-5912.
As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted in the Release Notes page of the Help Guide.
Release Notes — https://ninjatrader.com/support/help...ease_notes.htm
Note that if you are using Calculate.OnBarClose in a strategy, High Order Fill Resolution could be used when testing scripts on historical data. Selecting order fill resolution of "High" will allow you to set a secondary bar series, such as a 1-Tick bar series, to be used as the price data to fill your orders, this allows you to bring in more granular data then you are currently running the strategy on.
High order fill resolution: https://ninjatrader.com/support/help...FillResolution
Currently, Tick Replay and High Order Fill Resolution do not work together. If you would like me to I could add your vote to an existing feature request for using Tick Replay and High Order Fill Resolution together.Last edited by NinjaTrader_BrandonH; 05-04-2023, 12:09 PM.Brandon H.NinjaTrader Customer Service
- Likes 1
Comment
-
NinjaTrader_BrandonH We do already have the ability to enforce 1-tick intra-bar granularity by submitting the orders to a 1-tick series, do we not? And if we did so, the orders would always be filled basically as if "High" and "1-Tick" were used regardless of those settings, because we're putting the orders directly on a 1-tick series. This does not, though, address the original poster's concern that the bars are different in some cases.
- Likes 1
Comment
Latest Posts
Collapse
Topics | Statistics | Last Post | ||
---|---|---|---|---|
Started by Jimmyk, 01-26-2018, 05:19 AM
|
6 responses
835 views
0 likes
|
Last Post
by emuns
Today, 11:27 AM
|
||
Started by jxs_xrj, 01-12-2020, 09:49 AM
|
6 responses
3,291 views
1 like
|
Last Post
by jgualdronc
Today, 10:56 AM
|
||
Started by Touch-Ups, Today, 10:36 AM
|
0 responses
10 views
0 likes
|
Last Post
by Touch-Ups
Today, 10:36 AM
|
||
Started by geddyisodin, 04-25-2024, 05:20 AM
|
11 responses
62 views
0 likes
|
Last Post Today, 10:08 AM | ||
Can someone turn this thinkscript into a ninjascript. 5PM CST OHL and Previous Day
by Option Whisperer
Started by Option Whisperer, Today, 09:55 AM
|
0 responses
9 views
0 likes
|
Last Post Today, 09:55 AM |
Comment