Grain futures charts load with bad / incomplete data

[pfw_trader]

Hello,

I have been getting consistently 'bad' loaded charts in version 7 32bit (B19) for the ZC, ZS and ZW contracts (currently ZC 12-10, ZS 11-10, ZW 12-10). This is using Zen-Fire.
I have been using version 6.5 for live trading and am having no problems.

Snapshots of the charts (v7 and v6.5) are attached.

1. When the v7 charts load, there seems to be many missing ticks/bars.
2. There are wierd high 'spikes' on the last bar (60 min, 5 min and 50 tick). If you cannot reproduce this, try loading again during live grain market hours.
3. Viewing the data through the historical data manager, I cannot see the bad ticks for the spikes.
4. After right clicking on each chart and reloading all historical data the charts 'seem' to normalize.
5. However, when compared the the same chart loaded with 6.5, it can be clearly seen that there are many missing ticks and bars.
6. I have consistently seen this each day I load the grain contracts, it may exist for other contracts but I have not yet noticed them.

Thanks

[NinjaTrader_Kyle]

Hello pfw_trader,

Thank you for your post.

So that I may assist, I will need you to provide me with a few specific example bars. What I'll need is the timestamp and OHLC values for the bar you suspect is in error.

Once I have this information, I'll be able to compare to the results on our end and we should be able to move forward from there.

[pfw_trader]

Suspicious/invalid data loaded by NT 7 for grain contracts

Kyle,

After much investigation, reset of DB, instruments, reloading historical data, reinstall, etc.... this is what I found.

Using NT 7 - beta 19 and Zen-Fire feed:

1. The chart for ZC 12-10 is actually loading ZC 09-10 data. You can see this clearly in the attached graphic (as of 8/13/10 7pm EDT). The market analyzer shows the Last close and price for all the grain contracts for the next year period. The Market analyzer data seems correct as I checked it with CME Group site data.
2. The chart for ZS 11-10 is loading ZS 09-10 data.
3. The chart for ZW 12-10 is loading ZW 09-10 data.

Funny thing is that the data loaded on the charts with version 6.5 is correct. I deduce that the Zen-Fire/NinjaTrader DB for NT 7 historical data is different. There must be something incorrectly defined for the instrument month mappings, and/or the raw historical data contained therein is corrupt for the grain contracts.

This is also probably causing the strange spikes (see chart attachments in my first post) when the historical data that was loaded is then combined by NT 7 in the chart with the real-time tick data that comes from Zen-Fire (due to the spread that exists between contract months). When I check the Historical Data Manager / Edit - the tick data coming from Zen-Fire of course is not listed.

Hope this information helps. Of course, this prevents me from using NT 7 for live trading in the meantime until you are able to fix the Ninja Trader 7 historical data feed & instrument mappings.

Thanks

[NinjaTrader_Kyle]

Hello pfw_trader,

From the Control Center, please click Tools > Options.
In the options menu, navigate to the Data tab.

What is the 'Merge Policy' property set to?

[pfw_trader]

Kyle - it is set to 'MergeBackAdjusted'. The 'Get data from server' checkbox is checked.

FYI - is am also using ES, NQ, YM, CL, GC and all the FX contracts and they display/load fine for me.
Thanks.

[NinjaTrader_Kyle]

Hi pfw_trader,

Please switch your merge policy setting to 'DoNotMerge', then click OK.
Next, right click in your charts and 'Reload All Historical Data'.

This should correct your data display.

[pfw_trader]

Kyle, that fixed the chart - but now I don't understand how the merge & rollover works.

1. In the instrument editor. The ZS 09-10 rollover date is set at 7/30/10. Therefore, I expected that from 8/1/10, historical data from the next (ZS 11/10) will be loaded - and merged with ZS 09-10. Instead, historical data for 09-10 was loaded up to the very present minute for the 11/10 contract???? That's confusing.
2. Also, ZS 09-10 has an offset of -32-75 defined. However, this offset was not plotted in the chart when the merged data was loaded. Instead, the chart loaded 'non-offsetted' values.
3. The same is true for ZC 09-10 vs 12/10 contracts and ZW 09-10 vs 12/10 contracts.
4. Furthermore, if I now DO WANT to display a chart using a different merge policy, then do I have to override the global setting on the chart?

Thanks for your help.

[NinjaTrader_Kyle]

pfw_trader,

1.) The 11-10 contract has a defined roll-over date of 8/31. Due to this, data from the prior contract will be used as historical data up until the 31st of August when historical data from the 11-10 contract will be loaded.

2 / 3.) I'll need you to provide me with some specific examples of what values you were seeing vs. what values you were expecting in order to investigate.

4.) That's correct, the merge policy will need to be overridden and the charts will need to be reloaded.

[pfw_trader]

Kyle,

Thanks - that helps.

With regards to questions 2&3, using the 12/10 contract rollover date of 8/31, the merged data plotted is probably correct now.

1. If merge policy is used, should I then only expect the offset of -32.75 for the prior contract (09/10) be plotted on the 12/10 contract after 8/31?
2. Can I change the rollover dates? What are the dangers of doing so (e.g. getting overwritten after upgrade/reinstall/server reset).

[NinjaTrader_Kyle]

Hi pfw_trader,

1.) Correct - the offset applies only to the prior contract.

2.) You can change the roll-over dates from within the Instrument Editor's Misc tab.
This is saved as part of the NinjaTrader database. Thus, if the database is renamed, or if the software is reinstalled, this setting would need to be changed again.