Counter intuitive time stamps

[Harry]

Just to be sure that I am understanding things properly:

ES opens at 8:30 AM Central Time. I use a session template which splits the GLOBEX session into the night session (prior to 8:30 AM) and the RTH session (after 8:30 AM). Now if I look at the 8:30 candle (s)

Minute chart:

The 8:30 candle belongs to the night session.

Tick and volume chart:

The 8:30 candle belongs to the day session.

I assume that this is necessary because the milliseconds are truncated, but it is a real pain in case you code indicators, as you need to differentiate between the two cases.

[NinjaTrader_Kyle]

Hi Harry,

Thanks for your post and the subsequent clarification.

Yes, this is how things would function with that type of session template in place.

[Harry]

Bars.FirstBarOfSession Bug

Just found out that there is a related NinjaTrader bug that causes my indicators to produce false results.

The bug occurs on tick and volume charts. If you use a small period value for tick and volume charts, at session begin, there may be 6, 8 or more consecutive bars that have a time stamp equal to the session begin.

In the example below, there are 7 consecutive bars with a time stamp of 15:30:00 and NinjaTrader qualifies all of them as FirstBarOfSession. So this session has 8 bars that are FirstBarOfSession bars.

I have also checked for GetNextBeginEnd(). Each of these bar generates a full session on its own right. So the green area on the chart below does not only show 8 FirstBarsOfSession but also 8 complete sessions.

So the counter-intuitive logic of using the same time stamp (00:00) for different sessions as a function of the bar type, has already made its victims within NinjaScript.

[Harry]

Hi Harry,

Thanks for your post.

I think that this is the case, but I'd need to test on my end to know for sure.

Which session template are you referring to specifically?

You can test this with ES by using a session that divides the Globex session into night and RTH session.

[NinjaTrader_JoshP]

Harry,

For information on how bar timestamping works please see here: http://www.ninjatrader.com/support/h..._are_built.htm

On minute bars for 15:30, you contain data from 15:29:00 to 15:29:59. On tick bars, your 15:30 bar is the timestamp of the last tick in that tick bar. It is exactly as you guessed in that the milliseconds are truncated. The 15:30 minute bar does not contain anything from 15:30:00.01 while a tick bar could easily contain something like that and still be shown up as 15:30:00. So you can see how depending on the exact timestamp the bar would fall to before or after the session break time.


I've synced up to the exact time of your screenshot, but do not experience what you have highlighted. Please clarify how you have achieved this.

[Harry]

Josh, thank you for your answer.

I have taken your indicator and applied it to my chart. It exactly reproduces the problem.

I use NinjaTrader 7.0.1000.4

[NinjaTrader_JoshP]

Harry,

Even though our data sets already look identical, please export your data set and send it my way to ensure we can eliminate that as a variable.

Attached is the session template I was using on GMT-6. Modifying it to yours made no difference.

[Harry]

I am now on my second machine and just checking again.

Same problem, although I selected a different period and the data is not identical.

I will send you the exported data file via separate mail in a few minutes.

[Harry]

The problem was first discovered by another user from Argentine, so it is unlikely that it is related to country settings.

[NinjaTrader_JoshP]

Using your attached data with your session template in CET time zone. No dice on my end.

Can you try deleting (back up first) all of your chart cache (not data, just cache) and see what you get? This is located in Documents\NinjaTrader 7\db\cache.

[Harry]

This seems to be the right idea. I have deleted my entire cache and afterwards opened a new tick chart for ES. It was correctly displayed. Then I closed it and opened it again and the problem reappeared. So I made several tests:

(1) when the cache was completely empty and a chart loaded -> always correct sessions and only one first bar of session

(2) if after step one I closed the chart and opened a new chart again -> several sessions and first bars of sessions appeared

(3) toggling from ES 06-11 to ES 09-10 directly after step 1 -> ES 09-10 also had several one bar sessions -> toggling back to ES 06-11 -> also observed one bar sessions for ES 06-11 now

This means that the problem occurs whenever data is loaded from the cache. For correct display you need to delete the cache prior to loading of each chart.

[NinjaTrader_JoshP]

Are you connected to any data provider between any of those steps? What exactly is the data request range you are pushing out in the Data Series window? Merge policy set to MergeBackAdjusted?

[Harry]

Are you connected to any data provider between any of those steps? What exactly is the data request range you are pushing out in the Data Series window? Merge policy set to MergeBackAdjusted?

The data request range was 3 days. I am always in mode MergeBackAdjusted.

I am now offline and back on my main machine. Very strange as the chart that I have opened shows

-> only one session bar at the open on May 28 (chart 1)
-> but 8 one bar sessions at the open on May 27 (chart 2)
-> and 22 one bar sessions at the open on May 26 (chart 3)

This is the same chart, I just scrolled horizontally, and in one case every bar with the time stamp 15:30:00 is qualified as first bar of session and in the other cases it is not. Chart 1 is correct and this is inconsistent with charts 2 and 3.

[Harry]

Whether connected or not makes no difference.

However, if I display a data period for the first time, it is not loaded from the cache and will be correctly displayed. For example I just loaded the chart for May 20 which is a couple of days earlier and which was a period that I had not loaded previously, and the chart displayed correctly.

Then I toggled to another instrument and back to ES 06-11. Now the chart is corrupted.

So I am 100% sure and it is 100% reproducible on several machines. When the chart is loaded from the cache, you get false sessions. Where as, when the chart is loaded from the historical data base directly (first time loaded) it is correct.

Affected are all bars with the time stamp identical to session begin. In my case the session begin is 15:30, and therefore all bars with the time stamp 15:30 become single bar sessions. The problem can only be observed on tick charts and volume charts with small period values, as only these produce several bars with a time stamp identical to session begin.

Again, the problem has been confirmed by other users who alerted me to the malfunctioning on indicators relying on sessions.

[Harry]

If I change the lookback period of a corrupted chart from 3 days to 5 days, everything will be displayed correctly.

If then in a second step I toggle to another instrument and back to ES, the chart with the 5 day lookback period will be corrupted as well.