1 Year of Tick Data on ES

[lawyse]

I'm trying to backtest a strategy on a 10000 Volume chart of ES. Is it possible to get about 1 year of a continuous contract to do this? I love Ninja Trader, but backtesting volume charts hasn't worked at all for me. I can never do more than 3 weeks at a time. For this reason, I'm signing up for a TradeStation account until this can be improved. Any suggustions?

[NinjaTrader_Ray]

You would have to make continuous contracts using the merge feature however, volume bars are made up of tick data, backtesting 1 year of tick data will likely cause you to run into memory limitations.

More information on merging - http://www.ninjatrader-support.com/H...eV6/Merge.html

[ctrlbrk]

Ray,

Do you believe there should be a difference in memory consumption (and running out of it) depending on which data provider is used?

I noticed, for instance, that Zen Fire seemed to be much more efficient than IQfeed. I wondered if it was due to the nature of how NT was accessing the data.

[NinjaTrader_Dierk]

To clarify:
- if you mean download speed by "efficient", then yes, the down speed for historical data varies from provider to provider. As soon as the data is pulled from the provider and stored in your local hard drive, then it would not matter where the data actually came from.
- if you mean memory consumption on backtest, then this only is dependent on the actual number of ticks backtested. If would not matter where the ticks came from (= no dependency on provider)

[ctrlbrk]

No I really meant efficient as in the amount of time and memory (resources) NT takes to process the request. It seems both Zen and IQfeed max out my DSL connection of approx 700kb/sec, so I do not think one is transmitting data faster than the other.

I thought perhaps the external IQfeed app, vs the internal Zen, made a difference. You are saying no?

[NinjaTrader_Ben]

Hello,

I didn't review this entire thread but have you reviewed this link:

http://www.ninjatrader-support.com/HelpGuideV6/PerformanceTips1.html

[NinjaTrader_Dierk]

No I really meant efficient as in the amount of time and memory (resources) NT takes to process the request. It seems both Zen and IQfeed max out my DSL connection of approx 700kb/sec, so I do not think one is transmitting data faster than the other.

I thought perhaps the external IQfeed app, vs the internal Zen, made a difference. You are saying no?

Sorry, I don't follow. Not sure what use cases you are comparing other than the 2 I mentioned in my prior post.

[mgbloomfield]

No I really meant efficient as in the amount of time and memory (resources) NT takes to process the request. It seems both Zen and IQfeed max out my DSL connection of approx 700kb/sec

Whether or not Zen and IQFeed max out your DSL connection has nothing to do with NinjaTrader.

The download speed is specific to the data provider. NinjaTrader does not control nor determine the speed of the data download.

[ctrlbrk]

Whether or not Zen and IQFeed max out your DSL connection has nothing to do with NinjaTrader.

The download speed is specific to the data provider. NinjaTrader does not control nor determine the speed of the data download.

You've missed my point.

I've been using Zen a while longer now, and can say without a doubt it is much, much faster than IQfeed when it comes to chart-loading and backtesting.

Again, both providers maxed my internet connection. Yet one was much, much faster than the other.

So this means the way NinjaTrader handled the data (due to external API vs. internal, etc), or the type of data transferred, was significantly different from one to the other. I don't care about the specifics, I just care about the results.

[mgbloomfield]

You've missed my point.

I've been using Zen a while longer now, and can say without a doubt it is much, much faster than IQfeed when it comes to chart-loading and backtesting.

Zen-fire and IQFeed has nothing to do with the speed of chart-loading and backtesting. All data is loaded from the NinjaTrader database that is on your hard drive.

You're arguing that NinjaTrader is handling zen-fire's data feed more efficiently than IQFeed's data feed. I'm arguing the zen-fire's datafeed itself is more efficient.

It would be more accurate if you were to say that, "Zen-fire's data feed appears to be more efficient than IQFeed's data feed".

So this means the way NinjaTrader handled the data (due to external API vs. internal, etc), or the type of data transferred, was significantly different from one to the other. I don't care about the specifics, I just care about the results.

I care about results, too.

NinjaTrader is agnostic with data providers. There's no "external API vs internal API". Each and every data provider has their own API.

In the same way, NinjaTrader is agnostic with executing orders. Zen-fire, Trading Technologies, Patsystem, MB Trading, etc. each have their own API for executing orders. There's no "external API vs internal API" for executing orders. If someone were to discover that Zen-fire's execution engine is faster than it is simply faster because Zen-fire's execution engine is more efficient and nothing to do with NinjaTrader.

[ctrlbrk]

I am not arguing with your point of which data feed is more efficient. My original question was intended to raise exactly that. I do not agree on all things you've said, but it is not relevant to the thread.

NinjaTrader support seems to not know enough about it to answer or explain results. That is the main source of disappointment.

I find it hard to believe they have not drawn any conclusions on the efficiency of one provider vs. another with their program (again, regardless of where the efficiency differs, I am simply interested in the end result -- efficiency ie faster results or ie less resources [which means less NT crashing]).

Case in point, I was never able to load 2 months of tick data for backtesting with IQfeed. I have been able to do it with Zen before NT crashed. There are so many memory leaks and issues with Ninja that it only works rarely, but the most I ever got out of IQfeed was around 2 weeks if I recall correctly, so that is a huge increase in efficiency.

[NinjaTrader_Dierk]

>> Case in point, I was never able to load 2 months of tick data for backtesting with IQfeed. I have been able to do it with Zen before NT crashed. There are so many memory leaks and issues with Ninja that it only works rarely, but the most I ever got out of IQfeed was around 2 weeks if I recall correctly, so that is a huge increase in efficiency.

Please contact IQFeed support to clarify how far back they would have historical tick data available (I believe 2 weeks, but I'm not sure). NT does not control that.

[ctrlbrk]

Please contact IQFeed support to clarify how far back they would have historical tick data available (I believe 2 weeks, but I'm not sure). NT does not control that.

They have at least one year. You are not understanding the problem for some reason, still.

It is fine, I don't have time to keep going back and forth. I no longer have IQfeed. I also now realize it is futile to expect NT 6.5 to not crash or for NT support to fix anything, I am just waiting for NT 7 --- hopefully it will fix everything.

[paladincomp]

From the IQFeed documentation:

30 calendar days of tick (includes pre-post market) and several years of 1-Minute history (Forex back to Feb 2005, Eminis back to Sept. 2005, Stock/Futures/Indexes back to May 2007) retrieval for charting and time & sales data

[paladincomp]

I was comparing eSignal and IQFeed loading times with continuous futures contracts going back three days on an 8000 constant volume chart.

eSignal consistently loads the chart in about 10 sec after starting to receive data. IQFeed varied from 40 sec to nearly 60 sec.

Has NT been updated with the latest IQFeed API? It changed recently and some of the historical functions were deprecated...

just wondering why the big speed difference on my 26mbps cable modem.