Gap in historical data at midnight

[JS999]

When I start up NT beta 10, I notice that sometimes there is a gap in the data right around midnight. I have to choose "Reload Historical Data" and it goes away. This happens on all instruments on one particular installation of NT that I am running, which is monitoring a number of different markets. Has anyone else noticed this?

[NinjaTrader_Jason]

Hello JS999,

Could it be you experience a disconnection at the time the gap starts? Do you see anything in the Log-tab of the Control Center around this time?

You mention that it occurs on a particular installation. Am I correct you run NinjaTrader 7 on other machines that do not experience the issue?

If so, is there anything different regarding this particular installation compared to the others?

[JS999]

Hello JS999,

Could it be you experience a disconnection at the time the gap starts? Do you see anything in the Log-tab of the Control Center around this time?

You mention that it occurs on a particular installation. Am I correct you run NinjaTrader 7 on other machines that do not experience the issue?

If so, is there anything different regarding this particular installation compared to the others?

Actually when it is running I think it works fine. It's when I have it switched off overnight and then start up again in the morning that I see this, so it's not a disconnection issue, it's an issue with loading historical data on startup. I think I have seen it on more than one machine.

[NinjaTrader_Jason]

Ok, when you start NinjaTrader in the morning and do you not connect to your data feed yet, does the chart reflect the data that was present as per the previous session?

If you now connect to your data feed and data is loaded, a gap appears?

Could you please provide me reproducible steps so I can test it on my end.

[NinjaTrader_Dierk]

JS999: Harry from our forum keeps reporting similar (?!?) issues. Can't find exact link right now, may be this one:

http://www.ninjatrader-support2.com/vb/showthread.php?t=26401

Unfortunately we haven't been able to reproduce. May be there is value in sync'ing with him in order to isolate the "missing piece".

[JS999]

Ok, when you start NinjaTrader in the morning and do you not connect to your data feed yet, does the chart reflect the data that was present as per the previous session?

The previous session's data is there. I have my sessions set to Default 24/7.

If you now connect to your data feed and data is loaded, a gap appears?

Correct. There is a gap right around midnight. I have seen this on more than one installation. It definitely occurs on Range bar charts, but I think I have also seen it for minute-based charts too. There is simply some data missing between the start of the session and whatever the current time is (let's say today it was 7:30 AM when I turned it on). So I will see some data which begins after midnight and goes to the current time, but some data right after midnight is missing. When I hit Reload Historical Data, it appears. All of the data from previous sessions is there.

[JS999]

Incidentally, try loading data for GC 04-10. There is a gap at midnight on 3/3 that is still there and persists even after reload.

[NinjaTrader_Dierk]

What timezone is your PC?

[JS999]

What timezone is your PC?

Eastern standard time. I also have it sychronized with an atomic clock program that checks time with a number of internet servers every 2 minutes, so I know it's accurate.

[NinjaTrader_Dierk]

I just loaded GC 04-10, 1min, Default 24/7, fresh DB for the past few days. I could not see any gap at midnight 3/3. Could you please be specific? Thanks

[JS999]

I just loaded GC 04-10, 1min, Default 24/7, fresh DB for the past few days. I could not see any gap at midnight 3/3. Could you please be specific? Thanks

Dierk, tick-based charts and minute based charts actually load from different sets of data. So if you have a range-bar chart up and a minute-based bar chart up, and if you hit Reload Historical Data on one of them, the other will not load. They are being loaded from separate data sets on your server, obviously. The gap I noticed was using Range charts... try it again using a 10 range chart and let me know if you see it.

[Harry]

Yes I have hada similiar issue. The link cited below however is something different (daily bars are not correctly displayed between session end and midnight)

The thread on data gaps is here:

http://www.ninjatrader-support2.com/vb/showthread.php?t=25513

There are some similarites with the observations of JS 999, notably

(1) Gaps occur on connection
(2) Gaps disappear when reloading chart data

I made some further observations

(A) Gap may occur on one chart (example 5 min chart), but not on the other chart for the same instrument (example 30 min chart, same data base).
(B) When gap occurs, the data has been correctly loaded to the data base. This can be verified by editing the data via Historical Data Manager.
(C) Gap remains when changing chart template.
(D) Gap disappears when toggling to another instrument and back.

So in my case, it is not related to data loading, but to correctdisplay of the data loaded to the database. Points to some problem with internal synchronization of loading and displaying chart data, in particular when connecting to a workspace with several charts.

JS, can you confirm my findings (A) to (D) ?

JS999: Harry from our forum keeps reporting similar (?!?) issues. Can't find exact link right now, may be this one:

http://www.ninjatrader-support2.com/vb/showthread.php?t=26401

Unfortunately we haven't been able to reproduce. May be there is value in sync'ing with him in order to isolate the "missing piece".

[NinjaTrader_Jason]

Hello JS999,

I tested a GC 04-10 range (10) chart and I did not see any gaps, see screenshot.

To what data feed do you connect?

Can you please ensure both charts use the 24/7 session template.

[JS999]

Hello JS999,

I tested a GC 04-10 range (10) chart and I did not see any gaps, see screenshot.

To what data feed do you connect?

Can you please ensure both charts use the 24/7 session template.

I connect through Zen-Fire at Mirus futures. However, the historical data servers are the domain of NT, aren't they? Aren't we all loading historical data from the same source which is controlled by you guys? Anyways, here is a screenshot. I just reloaded the data, and it's still missing.

This is typically what happens when the issue I described earlier occurs, however in this case the Reload Historical Data option is not solving it.

[NinjaTrader_Jason]

Hello JS999,

I used Zen-Fire when I created the chart as well.

I tested it again, this time I used a 22 range interval, there is no gap on my end.

Can you please disconnect/re-connect to Zen-Fire. Try once more and check if the gap persists.