Problems loading data from IB

[tickling]

I worked some more with Beta 5 and IB. Most of the charts I use are ES minute data. More often than not, the request to reload all historical data just hangs. When I pressed the Abort key on the control center after a long time, I only got partial data. At some point this happened when Visual Studio was attached to NT and I got this:

An unhandled exception of the type System.NullReferenceException occured in System.Windows.Forms.dll
Object reference is not set to an instance of an object

then NT crashed.

I also think this issue may have to do with that.

[NinjaTrader_JoshP]

tickling,

Do you have reproducible steps we can follow for the null exception?

[tickling]

Well the setup is one chart of ES 1 minute for 15 days connected to IB with some innocent volume indicator in a panel. I do not have exact sequence of keystrokes but several historical reloads will cause IB to choke on pacing (it shows in the log). At some point, the reload just hangs. It happened to me that NT just dissapeared altogether. At the one time it happened with visual studio I could watch this message, but it does not seem to relate to visual studio. I will try some more and see if I can get more info about this.

[NinjaTrader_JoshP]

tickling,

Your log and trace files do not show any indication of the null exception. Are you sure you attached the files of the right date? Where exactly did you see this message?

[tickling]

Yes, the message was given by Visual Studio that was attached to NT7b5. The logs do show the pacing difficulty. When Visual Studio was attached it trapped the exception. When it was not attached, NT7b5 crashed without any message.

[NinjaTrader_JoshP]

tickling,

We are unable to reproduce NT disappearing. Please let us know if you are able to reproduce this without VS. Thanks.

[Harry]

Daily data not loaded correctly.

A simple test with a virgin NT 7 (control center only) setup.

When connected to IB, if I open a daily chart, this takes 16 seconds (reproducible). NT 6.5. takes 1.5 seconds (reproducible). I am talking of default charts with no indicator.

Something is still wrong in the way NT 7 accesses the daily data.

[NinjaTrader_Dierk]

I just tested NT7 and pulled daily data for ES in Tools->Historical data manager from TWS:
- 1st attempt -> 2 secs
- 2nd attempt -> 15 secs

-> IB response times are not predictable. You may consider subscribing to a different data feed in case you keep experiencing IB issues.

[Harry]

Thanks four your answer.

Made some more tries and observerd the same behaviour as you did, i.e. sometimes loading in 2 secs and sometimes loading in 16 sec.

However, NT7 is triggering pacing violations more frequently, when requesting daily data, see also other thread, where these pacing violations cause a freeze of the indicator menu and a crash of NT.

[NinjaTrader_Dierk]

I believe in the other thread your're referring to I provided some clarification an what NT could achieve and what not and that there will be some improvements with next beta.

[Harry]

I am referring to problem #1 of this thread http://www.ninjatrader-support2.com/...ad.php?t=23344

The problem has been reproduced by Austin, Bertrand and other users and forwarded to development (had some mails exchanged with Bertrand). It is about NT (connected to IB) locking up, when putting up an indicator on an intraday chart requesting daily data via

Data.Bars.GetBars(Bars.Instrument, new Period(PeriodType.Day, 1, Bars.Period.MarketDataType), Bars.From, Bars.To, .....

For example, when you add a standard pivot indicator while being connected, this can easily crash NT.

But as this is confirmed, I hope that it will be addressed by the next Beta.

[NinjaTrader_JoshP]

Harry,

To clarify, when you load pivots it will go off and request data. You will just have to wait for it to come back with your request. While it is off requesting data it is possible to have NT have the indicators window remain open. We were never able to reproduce NT crashing from such a request though.

[Harry]

Not invented

Josh, I believe that the issue is reproducible. Betrand told me that he has forwarded it to development.

This was the response from Austin:

Harry, I've tested this and can reproduce the issue (un-closable indicator window). We will look into this.

This was the response from Bertrand:

Harry,
Thanks much, think I got it this time here.

This was the experience of zweistein:

I did what Harry said and added a Pivot indicator to my ES chart. A big red X on the chart, but other charts still running. Must exit to resolve...
side effect: 20 days of ES wiped out in the data base

[NinjaTrader_JoshP]

Harry,

The unclosable indicator window is an effect from needing to wait for the data request to come back. We could reproduce this, but this is the way it will behave as IB returning data requests can sometimes be unpredictable as to when it will come back.

I don't think we were ever able to reproduce any red x issue though.

[Harry]

NT 7 not loading daily data as NT 6.5!

I had the red X only one time myself. Usually it is just the indicator window, which remains open and an empty chart behind.

Let me summarize again:

The whole problem only occurs, when connected to Interactive Brokers at the moment when daily historical data is loaded (i.e. when connecting, when opening a new daily chart or when adding indicators to an intraday chart that require daily data).

The problem only occurs with NT 7 but never with NT 6.5, so the problem here is NT 7!

This is what I have observed in detail:

(1) Loading of a simple standard daily chart (365 days) sometimes takes 16 sec., sometimes 2 sec with NT 7, but always 2 sec with NT 6.5. So NT 7 probably causes a pacing violation where NT 6.5 does not.

(2) When adding indicators on an intraday chart that call for daily data (standard pivots but also a number of proprietary indicators), NT 7 often locks up completely showing the indicator window. When this happens, you cannot abort loading or whatsoever, just wait and wait forever. The proprietary indicators and pivots I use for NT 6.5 and NT 7 are basically the same (with a minor change for Data.Bars.GetBars). Again for NT 6.5. this problem has never occured!

(3) Sometimes the indicator panel can be closed, but the indicators do not show up correctly. This also only applies to indicators that call daily bars on an intraday chart.

Conclusion: Compared to NT 6.5 it is clear that NT 7 loads daily data from IB in a different way

- causing more frequent pacing violations
- sometimes locking up completely showing the indicator window and an empty chart

This means that I cannot trade with NT 7 because - unlike for NT 6.5 - the main indicators that I use cannot be loaded!