Merging multiple contracts

[Harry]

Looks like a bug

Merged again. Popup reads as follows: - 'Day' data successfully merged. - 'Minute' data successfully merged. - No 'Tick' data available for 'From' instrument.

I used the period from 22.12.2008 to 09.09.2008 for ES 09-09, which was merged into ES 12-09.

If you look at the 1440 min chart, you can see that the merging procedure was successful for intraday data. However the daily chart shows data that has not been merged (although my data base contains daily data of ES 09-09 many months back.

I guess it does not work correctly.

[NinjaTrader_Bertrand]

Harry, so when you export the daily data from the ES 09-09 it's showing up as expected? Did you reload the 12-09 contract by any chance after the merge process?

[Harry]

Still no improvement

Hi Bertand,

My ES 09-09 daily data is displayed correctly. The merged contract is not displayed correctly - as you can see, there is no volume and liquidity is low causing gaps. Of course, I did not reload ES 12-09 from Interactive Brokers, as this would overwrite the merged data!

[Harry]

1440 min charts

Hi Bertrand,

I am now using 1440 min charts with a lookback period of 250 days. Not an ideal solution, but it works. If you still have an idea of how to merge daily data, I would appreciate.

[NinjaTrader_Ben]

Hello,

You can merge daily data. What is the issue you are having?

I am jumping in here, hope I'm not too off base.

[Harry]

Thanks for replying Ben. The issue is simple. When I merge future contracts at rollover date, the intraday data is correctly merged, whereas the daily data is not. I am using Interactive Brokers as a data supplier.

The daily data including volume is correctly displayed for a ES 09-09 chart, so it is definitely in the data base! If I merge this into ES 12-09 it simply does not work. The ES 12-09 daily chart does not show any merged data. However, the ES 12-09 intraday chart shows merged data.

One detail: When merging the popup reads (very fast, difficult to see) something like "Loading minute bars, merging, saving ...", but does not mention daily bars. Only in the end it wil confirm that day data and minute data was successfully merged.

I think this is a NinjaTrader bug.

[Harry]

Finally merging daily data worked

Finally resolved the issue. After repairing and resetting the data base I reloaded the last days of ES 12-09, merged historical data, and this time the daily data was also included. I do not know why, but the data is now correctly displayed.

[Harry]

Resetting the data base solved all problems

Also managed to merge all other contracts that I could not merge during the last three weeks. I did not need to reload any data, so the only thing that can have caused the problem is a corrupted data base, which ever causes so many problems. I recall: The only thing I did was repairing and resetting the data base.

[Harry]

I Do Not Believe It

When I shut down and restarted NinjaTrader I had a very bad surprise. I had successfully merged and checked daily data of ES, YM, NQ, TF, 6E, 6B, 6J and ZB. Out of these for 6E, 6B , 6J and ZB the merged data that I had seen and checked disappeared and the non-merged data reappeared !!! I tried to repair, reset the data base and merge these contracts again. Impossible. However, for ES, YM, NQ and TF the merged daily data was maintained.

I do not know what demons are working inside the NT data base, but some of them are particularly malicious.

[Harry]

There seems to be a solution

Upon each connection with Interactive Brokers, NinjaTrader automatically overwrites the merged data with unmerged data, whenever a daily chart is opened while connected. This explains why some of the contracts showed merged and other unmerged data.

Is there any way to tell NinjaTrader not to automatically backfill and overwrite the merged contracts daily data? I do not want to disconnect each time I want to have a look at a daily chart!

[NinjaTrader_Kyle]

Hi Harry,

Thank you for your reply.

Correct, upon reconnection, NinjaTrader will request historical data for any open charts.

Historical data will, in some cases, overwrite existing merged data.


Unfortunately, in the current version of NT, the only work-around for this is to disable historical data for the connection.

NinjaTrader 7 will not exhibit this behavior.

[Harry]

NT completely unpredictable

Actually what I do not understand here is: For the same chart, for some contracts the merged data is overwirtten by NinjaTrader and for others it is not. For example NT allows me to keep my merged data for 6E and 6J, but not for 6B. In an similar way I may keep NQ but not ES. The behaviour of NT is completely unpredictable here. That's why I call it the database demon.

And worse, often the merging of daily data does not work at all!

[Harry]

Or to be more precise: The merging process worked, the merged data is there, but the chart can not access it. I just had the experience again that there was no merged data on the chart visible. But when I closed down NT and restarted it, all of a sudden the merged data was there ( I was not connected during the whole process).

[NinjaTrader_Jason]

Hello Harry,

Please see the link below that displays when historical data is loaded from the data feed and when from the database.

http://www.ninjatrader-support.com/HelpGuideV6/WhenDoesNinjaTraderLoadHistoricalData.html

[AynRandFan]

There is simply no reason why merging data needs to be this complicated of a hassle -- creating a separate 5-year file of tick-by-tick data for back-testing shouldn't ever have been any more complicated than a few mouse clicks.