Historical Data loading with IB

[Harry]

NT 7.0.0.15 with TWS 900.6

Problem occured around 00:30 CET, when opening a FOREX chart (IB).

When loading historical data by using a simple 30 min default chart, NinjaTrader often made two consecutive data requests, one for the period from May 14 (last Friday, when I had last updated historical data) to May 18, and then a second one for May 18. As two requests were made for the same instrument and the same period, this triggered data throttling by IB, so loading the most simple chart took about 15-20 seconds.

This is what I observed in detail:

- two consecutive requests with data throttling: all FOREX pairs, NQ, TF, YM

- two consecutive requests without data throttling: all currency futures, CL, DX, FDAX, GC, ZB

- only one request: BC, FESX, SI

This does not make sense, below is the example for YM, you can see that the second query caused an additional waiting time of 13,1 seconds.

2010-05-18 00:56:52:453 (Interactive Brokers) IB.HistoricalDataRequest.Send: last=18.05.2010 instrument='YM 06-10' from='2010-05-14' to='2010-05-18' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2010-05-18 00:56:52:453 (Interactive Brokers) IB.HistoricalDataRequest.Send: request='YM;FUT;201006;0;;;ECBOT;;USD;;;1 min;5 D;0;TRADES'
2010-05-18 00:56:54:125 (Interactive Brokers) IB.HistoricalDataRequest.Process: itemCount=5636 instrument='YM 06-10' from='2010-05-14' to='2010-05-18' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2010-05-18 00:56:55:343 (Interactive Brokers) Data.Bars.GetBarsNow: instrument='YM 06-10' from='2010-05-18' to='2010-05-18' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0 session='CME US Index Futures ETH'
2010-05-18 00:56:55:343 (Interactive Brokers) IB.HistoricalDataRequest.Send: last=18.05.2010 instrument='YM 06-10' from='2010-05-18' to='2010-05-18' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0
2010-05-18 00:56:55:343 (Interactive Brokers) IB.HistoricalDataRequest.Send: request='YM;FUT;201006;0;;;ECBOT;;USD;;;1 min;5 D;0;TRADES'
2010-05-18 00:56:55:343 (Interactive Brokers) IB.HistoricalDataRequest.Send: waiting 13,1 seconds
2010-05-18 00:57:10:062 (Interactive Brokers) IB.HistoricalDataRequest.Process: itemCount=5637 instrument='YM 06-10' from='2010-05-18' to='2010-05-18' period=1 Min splitAdjusted=False dividendAdjusted=False bars=0

[NinjaTrader_Bertrand]

Hi Harry, thanks for your report - how do you determine you ran into data throttling by IB? I did not see any pacing violation in the log excerpts you've posted. Likely the difference in requests is related to the data amount you've stored for these instruments in your local database - best would to retry with no data available for those, the same would apply to your cached data in the DB folder.

In addition, please forward us your trace and logs via Help > Mail to Support we can review events in the time period of your issue.

Thanks

[Harry]

I have edited the post below and highlighted the data throttling. This caused a loading of a simple chart with no indicators to be 17 seconds.

Problem is that you cannot make two querys for the same instrument and period without waiting time.

[NinjaTrader_Bertrand]

Thanks for clarifying Harry, let me check on my end here.