data issues on continuous contract offset

[rrasche98]

Hey guys,

I loaded one-minute historical data from PI Trading for the ES contract (CME Globex) going back a few years to test some LT strategies since Interactive Brokers only provides limited numbers of months...

The front month data ties out but starts diverging for older months - I assume due to differences in the rollover offset adjustment...?

I looked at the Contract Months data for ES in the Instrument Manager. Are the rollover dates and offsets for each contract updated every time you log into NT? I found that contracts from 6-13 thru 9-11 had offsets of -5, -6.25 etc. but they were zero starting 06-11 for the next five years. Is that right?

In any case, the data that was in NT (sourced from IB) and PI Trading diverges already in March 2013... so the zero value offsets wouldn't be the issue anyways.

Any insights into fixing this problem?

Thanks!

[NinjaTrader_Matthew]

Hey guys,


The front month data ties out but starts diverging for older months - I assume due to differences in the rollover offset adjustment...?

Depending on your merge policy, this is likely the case. By default it will be to MergeBackAdjusted and will apply the rollover offset.

We have an indicator located below which will help you visualize this:

http://www.ninjatrader.com/support/forum/local_links.php?catid=4&linkid=558

I looked at the Contract Months data for ES in the Instrument Manager. Are the rollover dates and offsets for each contract updated every time you log into NT? I found that contracts from 6-13 thru 9-11 had offsets of -5, -6.25 etc. but they were zero starting 06-11 for the next five years. Is that right?

When you connect to the NinjaTrader historical data server, we will attempt to download the Offset values. If they do not exist on the data server, they will be calculated locally from the data in your database.

These settings are only downloaded when the "Offset" field is left blank or the rollover date matches the definition on the server.

Please let me know if you have additional questions.

[rrasche98]

Yes, I am using the default (MergeBackAdjusted) to merge the data.

But I still don't understand why the data is different beyond 1-2 months. The data for the ES contract is the same for April 2013 but different for March 2013 - the IB data is higher by $4.75 compared to the data from PI. Shouldn't both sets of data have been applied the same offset? Or is this an incorrect assumption?

Thanks

[NinjaTrader_Matthew]

The last rollover in the collection was 3/7/2013

You should see the 06-13 data as is for everything beyond that date as we're currently on that expiry.

Everything before 3/7/2013 will have the offset applied.

You may disable the offset, but keep the merge policy by going to Tools--> Options--> Data tab and set your merge policy to "MergeNonBackAdjusted"

[rrasche98]

Thanks, Matthew.

I do not want to disable the offset since I need it for back testing.

Regarding your last post... Yes, everything beyond 3/7/2013 stays as is. But my question is why do I get a difference between IB data in NinjaTrader (downloaded thru the Historical Data Manager) and PI data in NinjaTrader (imported thru the Historical Data Manager) prior to 3/7/2013? I guess my question is: shouldn't the offset be applied to both sets of data?

Another strange thing... the volume numbers are exactly the same beyond 3/7/2013 but then diverge - PI volumes are MUCH higher then IB volumes - on the data set before 3/7/2013...

Beyond the specific question to understand how the offset is applied to downloaded data vs imported data I was jus hoping someone had gone thru a similar data issue before and could share some pointers... Any help on this is much appreciated.

Thanks

[NinjaTrader_Matthew]

How are you comparing this data? Do you load it in a chart?

Is the data you download from IB, and the historical data you imported from PI in the same instrument? Or did you create a second instrument to compare the data to the ES data you get from IB?

[rrasche98]

PI Trading has a script that adjusts the ASCII data into the NinjaTrader format and appends "NT" to the name of the instrument. So the PI data is in a contract called ESNT. The IB data is organized the usual way under each quarterly contract: ES 03-13, ES 06-13, etc.

To compare, I go to the Historical Data Manager and select a particular date under ESNT (the PI data) and then choose Grid/Export to Excel. I repeat the same for ES 06-13 (the IB data) and then compare the two sets in Excel. Can you recommend a better way to do a quick comparison?

Hummmm... as I am writing this reply I realized that maybe NT doesn't "recognize" the ESNT data set as an ES contract and therefore doesn't apply the same offset being used for the ES 06-13 contract?

Thanks!


NEW EDIT:
There is a stock in the Instrument Manager with symbol ESNT. Is this something that was created by NT when I imported the PI Trading data set? Do I need to adjust the settings here after making this a futures contract?

[NinjaTrader_Matthew]

Thanks for the update - I think you're on the right track.

We have a handful of offsets we have on our servers and we check them based on the master instrument name. We would not recongize an instrument called ESNT and would not download the offsets.

However, if we cannot download them, we should attempt to calculate these based of the data in your database, although you will need to configure the actual rollover dates for us to calculate that.

Lastly, if this was imported as a stock, there would be no option for the rollover offset. There is not a way to switch a Stock type to a Futures type, so you will need to recreate the symbol and ensure you have set it up as a "Futures".

[rrasche98]

OK, thanks - I can definitely try that approach. Do you know if I can recreate the ESNT symbol while keeping all the historical data from PI (and avoid having to re-import everything again)?

[NinjaTrader_Matthew]

You would need to import it again.