Bug: 1 min bar mini-Dow import of GMT+0 data is converted wrong by NT7 for old dates.

[ChiTrader2000]

Hi,

When I import the attached mini-Dow Jun08 1 minute bar data (for only 1 date, June 5, 2008), which is in UTC time (straight GMT time, GMT+0), into NinjaTrader, I set the "Time zone of imported data:" field to "(GMT) Greenwich Mean Time : Dublin, Edinburgh, Lisbon, London".

The data imports successfully, but it is offset by GMT-6 hours instead of GMT -5 hours. On June 5, 2008, the time in Chicago was 5 hours less than GMT, not 6 hours less than GMT, due to Daylight Savings Time.

I have also attached the Custom Session Template I am using for the mini-Dow product in my NinjaTrader.

I believe that the NinjaTrader 1 minute bar data import needs to consider the calendar date of each 1 minute bar being imported, in order to get the hour offset from GMT time on that date, not the hour offset from GMT time on today's date (which is Nov 6, 09, the date I imported this data).

FYI, I am using NinjaTrader7 Beta 4, 32-bit version.

Thanks in advance,

ChiTrader2000

[NinjaTrader_Dierk]

a) these are standard .NET conversion routines. This is not controlled by NT
b) Each and every timestamp is converted on it's own
c) London timezone definitely has daylight saving time as well

You could try a different timezone to see if that made a difference.

[NinjaTrader_Dierk]

I just added "UTC" to the list of available timezones. This fix will be available with next update. Thanks for bringing this to our attention.

[ChiTrader2000]

NinjaTrader_Dierk,

Thank you very much for adding "UTC" to the list of available timezones! This is just what I needed.

You made a good point about the "(GMT) Greenwich Mean Time : Dublin, Edinburgh, Lisbon, London" timezone. I mistakenly took that timezone to be "UTC" because I didn't see "UTC" on the list, but as you correctly said, London does have daylight savings time as well.

Looking forward to the "UTC" option! Thanks again!

ChiTrader2000

[ChiTrader2000]

Issue: Imported Historical Data Not Converted to Time Zone of a Product's Session

Hello,

I need to import 1 minute bar data into NinjaTrader for 100's of products (1000's of individual contracts) that trade on exchanges in many time zones around the world.

I would like to be able to import the data with a single button click, but NT7 requires that a TimeZone for the imported data is declared for each import.

I have already created custom "Sessions" (TimeZone + SessionOpenTime + SessionCloseTime) with the proper Time Zones, and have applied those time zones to custom futures products.

Question #1: Do I need to do 1 import for every time zone separately, or can I do the following:

.................
1) Create 1 minute bar data files in Central Standard Time, converting the times from the native timezone to CST in the files.

2) Import the 1 minute bars with a declared Time Zone of "Central Standard Time".

3) Hopefully then, NT7 would identify the TimeZone of the product, and convert the data into times for the TimeZone of the product.
...............
I have tried the above procedure for a product trading on the Sidney Futures Exchange, and the problem is that the data that appears on the chart, as well as the data that appears in the (very nice) new Historical Data Editor is still in CST time, and the times have the same times as in the files, even though I applied a Session to this product that had the proper Australian Time Zone.

Question #2: What effect is declaring a TimeZone during the historical data import supposed to have on the imported data? I can't figure it out.

Question #3: What should I do to get all my multi-timezone data into NinjaTrader with 1 button click?

One option you may want to consider: In addition to adding "UTC" to the historical data import options, as described in NinjaTrader_Dierk's post below, could you add an option like "Data Is In Time Zone of Product's Session"? This could tell NinjaTrader to simply import the data as is, without any conversion.

[NinjaTrader_Dierk]

Let's not over complicate things:
- timezone would indicate the timezone of THE DATA in the file
- you needed to have one import per timezone IN THE DATA files. There is no way around

[ChiTrader2000]

NinjaTrader_Dierk,

Thanks for the quick reply.

Just so I understand 100% correctly, if I want to properly import an SFE product, I should do 2 things:

1. Create the 1 minute bar file with timestamps in the proper Australia TimeZone times.

2. When I import the data, I should choose the proper Australia timezone time in the "Import Historical Data" screen.

Are these the exact two steps I should take, or is there something else I should know?

Thank you very much in advance!

ChiTrader2000

[NinjaTrader_Dierk]

Not sure how to make it any clearer: you needed to select the timezone the timestamps of your data file(s) would be in.

[ChiTrader2000]

NinjaTrader_Dierk,

OK, thanks. I understand.

I just wanted to make sure, because the stakes are pretty high for me.

I needed to change a 1 minute bar generation that I will ideally run all night tonight during non-trading hours, and I will only get one or two more chances at it before I may have to wait until next weekend to try the 1 minute bar generation again. Unfortunately, I have a hard deadline this week, which means that next weekend is too late... that's why I asked what may have seemed like a silly question just to make absolutely 100% sure.

Since per your information, NinjaTrader data imports require that each timezone be imported separately I set up my program to dynamically generate folders for the timezones of products whose files will be generated, then generate the files for each product in the proper folder. That way, a user can be told to load the products in, one timezone at a time, and they can use the folder name to help them pick the proper time zone on the historical data import.

Thanks again for the help.

ChiTrader2000

[ChiTrader2000]

BUG (100% Sure): Import of IR 06-08 with correct timezone ticks & combobox -> CST

Hello NinjaTrader Development Team,

I have discovered a serious and reproducible bug in the Import Historical Data tool of NT 7 Beta 4.

Even though I created a IR 06-08 (SFE 90Day AusBAB) file in "(GMT +10:00) Canberra, Melbourne, Sydney" time and imported the data in the Historical Data Manager --> Import Tab as "(GMT +10:00) Canberra, Melbourne, Sydney ", the data is still converted to Central Standard Time.

I have attached files to help you debug this problem:

1. IR 06-08 Two Day.txt : The data, which is in "(GMT +10:00) Canberra, Melbourne, Sydney" time.

2. Historical Data Mgr Import Screen.JPG : Showing how the Historical Data Manager's Import screen looks right before I import the file, with proper time zone selected.

3. Hist Data Editor - Two Days Data Showing CST Bug Smalll IMG.JPG : Shows how the data is converted from Australia time to CST time.

Please note how the highlighted row's time is 5:28 PM, yet the timestamp of the same row in the text file is 8:28 AM. (The row's text in the text file is this: "20080417 082800;92.06;92.06;92.06;92.06;321").

4. Screenshots of the product's Intrument Manager and Session Manager.

Thank you in advance,

ChiTrader2000

[NinjaTrader_Dierk]

>> the data is still converted to Central Standard Time
I suspect your PC is central time no? -> expected

Again:
- all the imported data is converted to your local time zone
- timezone of instruments/exchanges are completely irrelevant
- you need to select the timezone of TIMESTAMP IN THE DATA FILE on import

Not sure how to make it any clearer.

[ChiTrader2000]

NinjaTrader_Dierk,

Now I see. You are 100% right. There is no bug... in fact, my confusion was due to a major improvement to NinjaTrader 7 which I just couldn't get my brain around, which is:

In NinjaTrader 6.5, timestamps of 1 minute bars are imported as-is, without changing the timezone from say Sydney time to my local timezone (CST).

In NinjaTrader 7, timestamps are converted from other timezones to my timezone, which my traders are going to love. Now, instead of having to look at the NinjaTrader charts of backtest results of Sydney products in Sydney time, my traders can look at the results in Chicago time, which is how they think of the trades in real life.

This brings me to another difference between NinjaTrader 6.5 and NinjaTrader 7 that NT7 upgrade users should be aware of, which is the reason that I started making my confused posts in the first place... NinjaTrader 6.5 did not merge the imported 1 minute bar data for contracts, whereas NinjaTrader 7 by default merges the imported contracts.

Since I didn't realize this, I incorrectly assumed that the reason that I had 2.5 months of missing data in the middle of an SFE AusBab 06-08 chart (the data was correctly displayed in the historical data editor, just not the chart) was due to the 1 minute bar data's time zone conversions, when in actuality it was due to the contract merging. By changing the global setting to "DoNotMerge", the charts immediately reflected 100% of the 1 minute bar data that can be viewed in the historical data editor.

I thank you very much for your time and instruction, and I apologize for the frustration my posts caused. The combination of two NT 6.5 --> NT 7 changes/improvements both affecting a chart at the same time made me come to incorrect assumptions about the reason for a lack of data in the middle of the chart.

Best regards and thank you again,

ChiTrader2000