Historical Chart Load Frustration

[Bogomir28]

Hi, i'm fairly new to NT having decided to come across from another platform to explore the potential of using C# to develop strategies.

At this point i'm just trying to get my head around the loading of historical charts and whether the frustrating wait times i'm experiencing are normal or not.

If I load 30 days of ES ## ## expiring 1st Nov 2011, in 7 tick Renko format, with no indicators, it takes several minutes to load. This seems like a long time, but if it needs to download the historical tick data, thats fine. Which it apparently does - at 300kb/s for a few minutes.

However, i would then expect that the next time i load my session, the same chart should come up relatively quickly, having already downloaded the tick data. Which it doesn't - it goes through the same download process.

Why does my NT seem to keep needing to download the same historical data? Am i doing something wrong? Is data downloaded via a chart the same as manually using the 'Historical Data Manager'? I have tried downloading relevant chunks of tick data using the manager, and yet any charts i try to load within that period still take ages and still do their own lengthy download before displaying.

The problem gets exponentially worse if i try to load a larger period - 90 days for example. I have to load the chart 20+ minutes before i want to start looking at it, and every time it downloads a large amount of data - regardless of whether or not i've loaded that exact chart before.

Any help would be much appreciated!!!

Thanks, Damien

[NinjaTrader_Kyle]

Hello Damien,

Thanks for your post.

Unfortunately, the behavior you've outlined here is as I would expect. If any section of a data set is not present, NinjaTrader would reload the entire data set. A change here is something development is looking in to for future releases, but I'm afraid that this will continue to occur in the current version; the only way to improve load times as it stands would be to reduce what data is requested.

[Bogomir28]

Hi Kyle, thanks for your reply.

By entire data set, do you mean a full-term contract? For example in the ES, if i downloaded tick data for the entire of the December 2011 contract would i then be able to view periods within that without needing to re-download anything?

Either way, it is good to know that what i'm experiencing is 'normal'. With time hopefully i will get a feel for the best ways to work around the issue. Definitely though i would think it an important area to address in any forthcoming releases.

Thanks again
Damien

[NinjaTrader_Jason]

Hello Damien,

Yes, if you load historical data for the entire December contract you will be able to view all data. However please check what session template is selected in the Data Series menu (if you download data via a chart). The session template determines at what hours you load data. So if the session template only loads regular trading hours (9:30 AM - 4:15 PM EST), it will not load data outside these hours. You will not be able to see this data in NinjaTrader either.

You can view the historical data you have present in NinjaTrader at Control Center-->Tools-->Historical Data Manager-->Edit-tab.

[Bogomir28]

Hi guys, thanks again for your ongoing help.

Basically NT just takes a while to load backdata sometimes, which is fine i guess.

However i'm still having issues getting NT to save data that i've downloaded. If i go to the download manager 'edit' tab like you describe, i can see that i have tick backdata for the ES ## ## going back to the 7th Oct 2011.

So, i go along to the download tab and set it to download tick data for the ES ## ## from 1st Sept 2011 thru 8th Oct 2011. Sure enough it starts downloading away at 300kb/s for a while, then says 'loading', and finally 'saving'. All good. However, if i then go back to the 'edit' tab, nothing has changed. If i restart the program - still nothing. And if i try to load up an ES chart for Sept 2011, it just comes up with a blank white page.

Frustrating! Any help much appreciated. Thanks, Damien

[NinjaTrader_Jason]

Hello Damien,

To what data feed do you connect in NinjaTrader?

Can you please send me your Config.xml file so I can test your connection on my end. You can find this file at (My) Documents-->NinjaTrader 7. Please attach it in your email you will send to support[at]ninjatrader[dot]com. Put 'Attn Jason, Historical Chart Load Frustration' in the subject header.

[Bogomir28]

Hi Jason, i am using a Kinetick feed with subscription to CME E-Mini.

I am sent you my config file, hopefully this can shed some light on things. In the meantime, things have gotten even more peculiar - NT seems to have deleted some of my tick history - I now have data only going back to the 20th Oct?? Where before it went back to the 7th Oct.

Is there a limit to how much history your can store?
Do you think a full reinstall would help?

Thanks!
Damien

[Bogomir28]

Hi again - so, a quick update: i have tried uninstalling NT and reinstalling, and downloading tick data one month back at a time to try and narrow down the issue.

Basically, it flat out refuses to download any tick data prior to the 24th Oct 2011. If i select the month of October, it downloads only the last few days. If i select dates prior to 24th Oct, it does nothing.

I am subscribed to an E-Mini feed from Kinetick, and trying to download daily tick data for the ES ## ## continuous contract.

Is there a limit on historical data for those using only using a 'free' version of NT? Or could the problem be with Kinetick? Should i try a different data provider perhaps??

Thanks for your time.

[Bogomir28]

So! Turns out that Kinetick only supply 120 days of tick data. Easy solve there!

So, my next question is going to be - how can i get access to a larger amount of historical tick data? Surely anyone wanting to seriously backtest a strategy is going to need at least a year or two of data.

Is there any way to get more history out of Kinetick, or can anyone recommend another data provider?
Many thanks.

[NinjaTrader_Jason]

We support also IQFeed, BarChart and eSignal as data feed providers. Unfortunately I am not sure how much historical tick data they offer. I suggest to contact them for clarification.

I am not aware of a way to load more historical tick data with Kinetick.

[Bogomir28]

Hi Jason, thanks for your reply. Kinetick's 120 days of tick data is actually the best out there it seems.

The only other option may be to purchase historical data from places like tickdata.com, which is in ascii format.

If it is possible to import this into NinjaTrader then no doubt there will be forum threads on it somewhere - however is there anything you can tell me off the top of your head? ie Have other members done this in the past, and is it a difficult thing to achieve?

Thanks!!

[NinjaTrader_Jason]

Yes, you can manually import historical data as per the instructions at the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/importing.htm

The data needs to be in a certain format which you can find in the link above.

Other users have used this function successful in the past.