Trouble loading large amounts of data in NT8

[NinjaTrader_Jim]

Hello guillembm,

In order for data to be consistent between NinjaTrader 7 and NinjaTrader 8, the following must be considered:

1. PC clock time is the same as NinjaTrader 8 and TWS
2. Instrument definitions are the same between NinjaTrader 7 and NinjaTrader 8

^ The above needs to be the same before the data is downloaded in order to be sure that the data will align when exporting and migrating.

I performed a test when changing my PC timezone to Brussels, as well as in NinjaTrader 8 and Trader Workstation. I used TWS 978.2c with NinjaTrader 7 and TWS 981.3g with NinjaTrader 8 for this test, using AUDUSD which I have data for.

I first needed to update the instrument definitions in NinjaTrader 7 so they matched NinjaTrader 8. Once these aligned, my tests to download fresh data showed my charts to and data to align.

After this, I disconnected from IB and I further tested importing data that was exported from NinjaTrader 7, and to also test migrating the data.

These tests all show the data aligned with the exception of how NinjaTrader 8 will set volume for the FOREX data that it downloads.

This tells us that the software is working correctly and we can follow this import process or migration process when these factors are the same before the data is downloaded. (Data would need to be redownloaded with the above factors in mind and exported again in order for it to be aligned in NinjaTrader 8.)

Ultimately here, it may be very difficult to analyze all data on the NinjaTrader 7 platform to point out differences and to redownload parts that are not aligned so they can be reimported into NinjaTrader 8. This would be one path forward.

Another alternative would be to request the data anew after making sure the instrument definitions and timezones are aligned. This may be difficult with Interactive Broker's data pacing rules, so considering a dedicated market data provider may also be considered.

I have attached screenshots of my tests.

Let me know if there is anything else we can do to help.

[guillembm]

Hello Jim,

thanks for your detailed instructions. Loading all data anew would not be a good solution. It's a lot of time and a lot of instruments.

Instead, after checking that all the points that you mention in your post are correct,I opted by deleting all migrated data and forcing the migration again but, without reinstalling NT8.

It seems that it worked. Now data between both versions are consistent. I have been verifying several instruments and, so far, everything matches.

[guillembm]

I found some differences in the instrument's settings between NT7 and NT8. Probably it's not relevant but it's just to make sure.

As an example, in the instrument settings for the symbol AUDCAD in NT77 "Currency" is marked as USD and in NT8 is selected as CAD. The tick size is marked as 0.00005 in NT7 and 0.00001 in NT8.

Does it make any difference?

[NinjaTrader_Jim]

Hello guillembm,

If the data driving a strategy changes, the results will be different. If consistency between the NinjaTrader 7 and NinjaTrader 8 environments is desired for this symbol, I would suggest updating instrument definitions, checking charts to compare the data, and to reload data after making changes to the instrument definitions.

[guillembm]

I would like to understand what it means for the indicator values and for the trading results the difference between the values of the "Currency" item in the instrument definitions and the value of the tick size.

I think that using that a lower tick size could significantly affect the performance.

[NinjaTrader_Jim]

Hello guillembm,

Any difference in instrument definitions results in a difference in how the instrument data would be read. This would then change the bars of that data series, and then indicator plots will be calculated differently. The instrument definitions should be consistent and set up correctly. (Some NinjaTrader 7 definitions may be old in comparison to NinjaTrader 8)

Currency can also affect the actual symbol mapping in some cases. I have seen some stocks from London Stock Exchange use the same Interactive Brokers symbol mapping, but specifically using the correct currency makes a difference in fetching that symbol.

[guillembm]

So, should I update the NT8 definitions to be the same as NT7?

Nt8 default definitions aren't supposed to be more correct?

[NinjaTrader_Jim]

Hello guillembm,

NinjaTrader 8 receives instrument definition updates regularly while NinjaTrader 7 requires updates to the installer to carry new instrument definitions. I would suggest using the newer definitions found with NinjaTrader 8 and expecting these to be more up to date than NinjaTrader 7's.

If you change some instrument definitions, this will not update the data on your platform, so the data would need to be redownloaded to get data/bars with the updated definitions.