Bar construction when not using Break at EOD

[BarzTrading]

There's either a bug in v8.1.1.1 (64-bit, Windows 10, Kinetick data) or I misunderstand how bar construction is supposed to work:

Steps to replicate:

1) Open a new Chart
2) Select YM ##-## as the data source.
3) Choose 120 minute bars.
4) Choose Custom range of 1/1/2018 to 1/1/2020.
5) Choose Trading Hours template "CBOE US Index Futures RTH" <-- (8:30am - 3:15pm)
6) Uncheck Break at EOD

Starting with 1/1/2018, observe the bar times at the end of each day. For the first several days it all behaves as I would expect. If a full bar cannot be completed in a session, the last bar of the session will have whatever the time is of the last bar that fully fits in the session.

For example, suppose the last bar of the session ends at 2:30pm, 45 minutes before the 3:15pm session close.

On the next session/day, the bar time of the first bar is 9:45am, because it includes the 45 min from the previous session and adds another 1hr 15min to the 8:30am session open. This forms a bar that consists of 120 minutes total and spans the session break.

So far, this is what I would expect.

But as you continue forward in time on the data series, eventually (within 5 to 20 days), you get to a bar whose time is exactly 3:15pm, the exact end of a session. One would expect the next bar to begin at 8:30am of the next session and end at 10:30am. So, the first bar of the next session should have a time of 10:30am.

But instead, from here on out, every first bar of the session has a time of 9:15am (only 45min after the open). And the last bar of the session is at 3:15pm. The same thing repeats for the rest of the data series.

Notes:
* I purged all historical data and tried again and got the same behavior.
* I quit NT, rebooted and still got the same behavior.
* Same problem with 60min bars (Within a few days the first bar of each session is 9:15am, a 45min bar, and the last bar of each session is 3:15pm)
* Same problem with 20min bars. (Within a few days the first bar of each session is 8:35am, a 5min bar, and the last bar of each session is 3:15pm)
* This did NOT happen with 17min or 21min bars. They worked as I'd expect.

Hoping you can replicate this behavior or explain why it behaves this way.

Thanks,

[NinjaTrader_BrandonH]

Hello BarzTrading,

Thanks for your post.

The current version of NinjaTrader is 8.1.1.3 at this time. Please update NinjaTrader 8 using the steps below. Updates to NinjaTrader will contain the latest security updates and bug fixes, as well as occasional new features and instruments.

To update NinjaTrader:

• Shutdown all programs including NinjaTrader *important
• Download NinjaTrader from the link below
  • Click here to download NinjaTrader
• Double-click the downloaded file to execute it
• Follow the on-screen instructions
• Restart NinjaTrader

You can read about changes to this release in the release notes linked below.

https://ninjatrader.com/support/help...ease_notes.htm

That said, when the Break At EOD property is disabled the bar on the chart must satisfy to completion before creating a new bar, regardless of the time of day.

When the Break At EOD property is enabled a new bar will be formed during the new trading session before the prior bar has been completed. Your bars are ensured to have a known starting point of which to give you a consistent, repeatable chart when this property is enabled.

"the bar time of the first bar is 9:45am, because it includes the 45 min from the previous session and adds another 1hr 15min to the 8:30am session open. This forms a bar that consists of 120 minutes total and spans the session break.

So far, this is what I would expect."

To have your chart bars consistently behave in this manner, you should enable the Break At EOD property.

See this help guide page for details about how the Break At EOD property works and an example: https://ninjatrader.com/support/help...eod.htm

[BarzTrading]

I will upgrade and report the results. But did you try to replicate the issue? Is it a known issue? I'm not sure from your response whether you really understood what I was saying.

[NinjaTrader_BrandonH]

Hello BarzTrading,

Thanks for your note.

Yes, I was able to replicate the behavior you reported. To have your chart bars consistently behave in the manner that you had described in the initial post, you could enable the Break At EOD property.

When Break At EOD is enabled, the chart bars would behave as you are expecting them to.

[BarzTrading]

Hi Brandon,

I tested in v8.1.1.3 and the behavior is identical.

I still feel like you're not understanding what I'm saying. The way certain bars are being constructed when Break at EOD is disabled is wrong.

I'm not trying to get my bars to behave a certain way, I'm trying to report a bug.

With 17min and 21min bars, the bars are constructed correctly for the duration of the data series. With 60min and 120min bars, they are NOT constructed correctly.

I would chalk it up to my own misunderstanding if I couldn't clearly see that the 60/120 bars are built much differently than the 17/21 minute bars.

[BarzTrading]

I've attached a Word doc with screen shots where I try to illustrate the issue. Use Ctrl+MouseWheel to zoom as needed in order to see the image detail.

[NinjaTrader_BrandonH]

Hello BarzTrading,

Thanks for your note.

I am able to reproduce the behavior you are reporting simply by testing the YM 06-23 instrument with 60-minute bars and setting the End Date property set to today's date with 10 Days to Load on the chart, and using the Default Trading Hours template for the instrument. See the attached screenshot showing the Data Series settings used.

After investigating the behavior, the holidays that occur are throwing bar values off on the chart.

For example, there is an early close on April 7th (Good Friday) at 8:15AM. So with the holiday early close, there is a 7 hours and 45 minutes less for the trading day. The next hour when the session opens on Sunday starting at 4:00PM, is subtracting back 45 minutes to the bar closing at 8:15 the previous Friday. So from that point forward, all bar closes will have that difference of closing 15 minutes earlier.

The next holiday with an early close would add to that. So if the close is 7h45m early again, it is a difference of 30 minutes to the end of the hour bar.

You could consider creating a Trading Hours template that does not include holidays to get the view that you are expecting.

Creating a Trading Hours template: https://ninjatrader.com/support/help...ngHourTemplate

Please let me know if I may assist further.

[BarzTrading]

Thanks, Brandon. Unfortunately, you seem to have found a completely different issue. Your explanation does not appear to explain what I have reported.

It would help if you would use my exact scenario.

Let me be clear with what I'm looking for from this thread:

• I don't want a work around. That's not my purpose here.
• I want an acknowledgement that what I've found is a bug. OR I want an explanation for why it's not a bug.
• If we can agree that it's a bug, I would like to know that a ticket has been issued for the development team to consider.

I have simplified my scenario slightly by using a shorter time window. I will list the simplified replication steps here:

1) Open a new Chart
2) Select YM ##-## as the data source.
3) Choose 60 minute bars.
4) Choose Custom range of 12/15/2019 to 1/1/2020.
5) Choose Trading Hours template "CBOE US Index Futures RTH (NoHolidays)". OR just use "CBOE US Index Futures RTH". It's the same.
6) Uncheck Break at EOD

Now review your chart and compare to this image. Please answer these questions:

• do you observe the same bars and bar times?
• do you think this is a bug? If not, why not?
• will you issue a ticket for the dev team or validate a ticket exists?

Blue boxes are correctly constructed bar times. Red boxes are incorrect bar times.




As I said it's not important whether you use a trading hours template with no holidays for this example, but here's my template with no holidays:

[NinjaTrader_BrandonH]

Hello BarzTrading,

Thanks for your notes.

I have tested the exact scenario you have described using the same Data Series settings and the same Trading Hours template and I am not seeing the same behavior that you are reporting. The time of the bar values on my end is not the same as the screenshot you shared

See this demonstration video: https://brandonh-ninjatrader.tinytak...N18yMTI5Njg1NA

If I am not taking the same steps as you are in the demonstration video to reproduce the behavior you are reporting, please let me know what the step(s) is that I have missed.

That said, please test to see if you are able to reproduce the behavior on the ES instrument and the 'CME US Index Futures ETH' trading hours template with the holidays removed. This would make finding a possible issue simpler since the hours of the bar will end at the end of the hour.

I look forward to assisting further.

[BarzTrading]

Thanks, Brandon.

In the video it looks like you're doing everything the same way I am.

However, the bars we see are definitely different. They are similar, but not the same.

ES shows the same problem.

ES and YM both have problems even when using the (NoHolidays) template.

Using the ETH trading hours template also shows some weird behavior, but I haven't dug into further yet.

At this point is seems as though there is something unique to my system or instance of NT that is the problem.

I've deleted historical data through the GUI and reloaded it, but it makes no difference.

What would you suggest at this point to try to get my system working properly?

[BarzTrading]

Hi Brandon,

I'm not sure why your data doesn't match mine in the screenshot above,

I've now replicated this exact behavior in numerous scenarios:

1. Windows 10, NT v8.1.1.3 Upgrade. Kinetick data and NinjaTrader data feed. Both ES and YM.
2. Same machine, but I completely uninstalled everything and did a clean download and virgin install of v8.1.1.3, ensuring all previous folders in Documents and ProgramFiles were deleted or renamed. I found the exact same behavior as before for both Kinetick and NinjaTrader data.
3. Windows 11 laptop, NT v8.0.27.1. Installed several months ago. Kinetick data gives EXACTLY the same behavior as the screen shot above. Both ES and YM.

I've compared the 60min bars to 5min bars. The 5min bars are built correctly. It appears that with the 60min bars NT starts turning the first bar of the day into a 75min bar instead of a 60min.

I've reproduced similar behavior using a custom trading hours template from 8:30-15:00 which is 6hr 30min. The first time the bar close matches the session close time of 15:00, the next bar close time at the start of the next session has the wrong duration. In this example, it creates a 30min bar with a bar close time of 9:00 when it should be 9:30. It seems to adjust the size of the first bar of the session so that the remaining bars (which have the correct 60min duration) will result in the last one of the session closing at exactly the close time of the session.

I'm fairly convinced there is a problem with bar construction and that it's not just my system.

Yet I'm still perplexed by why you don't see it.

Would you mind bringing someone else in on this as a second set of eyes?

[NinjaTrader_ChelseaB]

Hello BarzTrading,

I am also unable to reproduce using a template that does not have holidays.

Please send an email to support [at] ninjatrader.com so that I may schedule a call where I may test these steps on your end, and analyze why the behavior is causing differences.