Loading lots of data

[blar58]

Hi all,

I have a problem of loading time with one indicator.
This indicator needs a primary data (15 minutes) and I have to load also BidAsk data.

It means that I have to add this to the initialize method :

Add(Instrument.FullName,PeriodType.Tick,1, MarketDataType.Last);
Add(Instrument.FullName,PeriodType.Tick,1, MarketDataType.Ask);
Add(Instrument.FullName,PeriodType.Tick,1, MarketDataType.Bid);

I have to load 50 days to apply the indicator.

It takes a lot of time ... many minutes to load !

Is there a faster way to load all this data ?

Thank you

[VTtrader]

blar58,

50 days of 1 tick data is HUGE, and you're calling it 3 times for no real reason, that I can see.
MarketDataType, is for real-time processing. You shouldn't be calling it for historical bars.

If you need 50 days worth of 1 tick data, then just call it once, that will get your data in a 1/3rd the amount of time. But don't use the MarketDataType in the call.

VT

[blar58]

Thank you VTTrader for your response

But if you look at intellisense you will see that you have access
to a method : Add(Instrument, periodtype, period, marketdatatype)

What I am trying to do here is to get the historical bid, ask and last so that
I can manipulate them to get the historical delta.

Am I on the wrong path ?

[VTtrader]

Sorry Blar58, I didn't mean to mislead you, I was thinking back in the days of NT6.5, when there wasn't the support for historical bid/ask data.

That said, 50 days is an enormous amount of tick data, just for the Last price, and the Bid/Ask series would be far larger. I'm not sure what you're trading but if it's an active instrument, that's a ton of data. Perhaps a better approach would be to calculate in smaller chunks and write/append it to a file. Then read the results of your calculations from the file.

You may also run into some alignment issues, since the bid/ask values will change more frequently than the last price, it will be tricky to sync them together accurately.

VT

[blar58]

Thank you VTrader

Yes I know about the sync issue.


I think that unfortunately this problem cannot be really resolved.
I would dream of the day that we can load 50 days of tick data in seconds !




Cheers