Bug in Chart Loading, Data Backfill, CL Contract

[shawnj]

NT: 6.5.1000.14
Zenfire.
Chart, Days Back: 3.

Frequency:
This happens every morning but this morning I decided to document it.

Background:
Each afternoon at about 3:15 PM Central (15:15 on the chart x axis) I exit NT for the day. Each morning about 8:00 AM I launch NT, login, and open my workspace that includes a CL (Crude Oil) chart and some other charts.

Problem:
Uppon loading the chart, CL chart does not backfill the data from 15:15 to midnight on the previous day (See attachment: CL Overnight Gap). I have to manually "Reload Historical Day" to get CL to backfill properly (See attachment: CL Overnight Gap after ReloadHistorical). None of the other charts, such as YM have this problem. They properly backfill the missing data from 15:15 to midnight.

Suggestion:
Please fix the CL chart loading so that data for the previous day is backfilled all the way up to midnight.

thanks,
shawnj

[NinjaTrader_Kyle]

Hello shawnj,

Thank you for your post.

1.) Just to confirm, NinjaTrader is being shut down at the end of the day - correct?

2.) Have you tried this test with a new workspace? If not, please create a new workspace (File > Workspaces > New Workspace) and let me know if you see the same behavior there.

I will work to replicate this on my end and will update you on my results tomorrow morning.

[shawnj]

1) Yes NT is shut down at the end of the day.

2) The attached screen shots were from a newly created chart this morning. Not the chart that is in my workspace. The gap is the same in both.

[NinjaTrader_Kyle]

Hello shawnj,

Thank you for your reply.

I am not seeing the same behavior on my end. Please send a note to Support [AT] NinjaTrader [DOT] com with 'ATTN: Kyle - 286490' in the subject line.

In the message, please include the following:

1.) A link to this forum thread.

2.) Your most current trace file.
You will find the trace file here: My Documents > NinjaTrader 6.5 > Trace > trace.YYYYMMDD.txt

[shawnj]

Ok Kyle. I'm going to do a little more fault isolating on my end first. In my last reply I said the screen shot was from a new chart which was correct but I still had my production workspace open. This morning, before I opened my workspace, I just directly opened a CL chart and It had backfilled correctly. Then when I opened the workspace, it was correct. I'm tentatively thinking, which you hinted at, this has something to do with this particular workspace. ugh.

I'm going to keep opening my production workspace first thing in the morning as usual, and when I see the problem, then I'm going to exit the workspace and exit NT (to start clean) then relaunch NT and then directly open a CL chart. If it backfills correctly, then I'll start trying to fault isolate this particular workspace. ugh. ugh.

thanks,
shawnj

[shawnj]

Some followup:
The data backfill gap problem happened again this morning.

Sequence of events:
Yesterday afternoon at about 3:15 PM Central I exited NT. This morning about 8:00 AM I launched NT, logged in, and opened my production workspace. The overnight gap was present in only the CL chart as was described in my original message. This time I closed the workspace and exited NT. Then I relaunched NT, logged in, and directly opened a CL chart. The CL chart HAD the overnight gap. The only way to fill the overnight gap was to "Reload Historical Data".

Action:
I guess I'll create a new production workspace and perhaps each morning add a new bit from my current production workspace and see if I can start isolating the problem.

Kyle, I'm emailing you this mornings trace file as you requested. I hope you can help me isolate this problem. I still think there is a bug in NT relating to this. Of course users should not be able to construct a workspace that causes data backfill problems when opening the workspace.

thanks,
shawnj

[NinjaTrader_Kyle]

Hi shawnj,

Thank you for your reply and the additional information.

I will follow up with you via email as soon as I see the message. Please remember to include 'ATTN: Kyle' in the subject line.

[shawnj]

More followup:

After a fair amount of trail-and-error and studying NT trace logs and studying workspace xml files, I believe I have isolated the conditions that cause the Ninjatrader bug that I'm documenting in this thread.

Last night I created a new, very simple workspace that consists of only two charts. There are no Indicators or Strategies. It is just pure Ninjatrader . This morning, this new workspace duplicated the overnight gap problem. The attach screen shot is from one of the two charts in this new workspace.

The two charts are set up as follows:
1) CL 04-10, 100 Range bar, 60 days back.
2) CL 04-10, 50 tick bar, 3 days back.

I think the key thing in the workspace that is causing the bug is the workspace loads the 100 Range bar chart first. The order of the loading is important. My guess is these very long bars straddle the overnight period and that is somehow "confusing" the code that determines how far back to ask for historical back-fill data. The trace log shows the first CL chart loaded only asked for data from 3/11/2010 (Today). To back-fill the gap, it needs to ask for data starting from 3/10/2010.

Without looking at the source code, I don't think I can isolate the problem any further.

Kyle, If you are still out there, I'm emailing you this workspace, and trace log. This trace log does show an Error. This error may or may not help identify the source of the bug.

Tomorrow I'm going to test what I think will be an acceptable workaround to this bug. My theory is the long bar chart (100 Range) should be loaded last.

thanks,
shawnj

[NinjaTrader_Kyle]

Hi shawnj,

Thank you for your reply.

I will follow up with you via email.

[shawnj]

And another followup:

I think I have confirmed by theory that this data back-fill gap bug that I am documenting in this thread is being caused by a large bar chart (100 Range) being loaded first in the workspace for that contract.

I changed the chart loading sequence in my original workspace by editing the workspace xml file (don't try this at home ). I moved the 100 Range chart down to the bottom of the file thus causing it to load last (It originally was the first of the CL charts to load).

This morning this edited workspace loaded all the data without any gaps.

I'm still calling this an NT bug since a commercial charting package should not be sensitive to the chart loading sequence.

thanks,
shawnj

[NinjaTrader_Kyle]

Hi shawnj,

Thank you for your reply.

Please see my email regarding renaming your database. If that does not do the trick, please follow up with me via email.