NT7b19 + IB-TWS906 = freeze (sometime...)

[Trendseek]

Hello Ninjas,

I am on WinXP+SP3 and made an update to the new nt7b19 together with your new supported TWS-version.
While switching between different Instruments within a Chart-Window, it occours that NT freeze while (re-)loading an Instrument.
In this Situation NT has a verry high cpu-load - also after I close the Chart Window.
If I did not close the Chart-Window, after a while IB gives me a Message that i get no Quotas for the next five minutes.
When I try to Exit NT it gives me an Error Message.

Unfortunally I found no way to reproduce that Error.

Best Regards
Trendseek

[NinjaTrader_Jason]

Hello Trendseek,

From you description, loading the chart data by switching instruments was quite heavy on your CPU. Upon switching instruments, NinjaTrader will load the requested data from your connectivity provider (if not present in your database). It will need to calculate any indicator you have applied in the chart as well. This seemed to put too much load on your CPU and temporarily resulted in unresponsiveness.

I suggest to reduce the indicators in the chart if you have multiple applied and reduce the amount of requested historical data. The chart is requesting quite some data since you run into the IB data limitation as well. Unfortunately IB has a data limitation that allows you to load only that much historical data in a certain period. Once you hit the limitation, you will receive the error message you describe and you will need to wait 5 minutes in order to load historical data again.

[Trendseek]

Hello Jason,

thanks for your explanation. This could be the Case - because I also run the Marketanalyser with five other Instruments. In Germany we have a saying, that someone dont see the wood in the midst of trees...

On the other Hand - I just checking some new Indicators in different Timeframes and for different Instruments. only for the last 3 Days - that wasn't very Data-Intensive... and the IB-Data-Limitations are not soooo much restictive. But I will check this, and take a closer look to the NT-specific Data-Requests.

But all that is no reason for the freezing. I am only use a few simple Indicators. Anyway I will give you a second report if this happens again.

Regards
Trendseek

[Lee123]

Beta 18, 19, 20 freeze

Hi,

this has been ongoing in prior versions as well.

When IB loses its data or has to login again, NT freezes (hangs). Windows XP "dump reg" then hogs the cpu and NT will only close via the XP task manager after I close the dump reg. NT will not reconnect. ( NT connects to IB via a network)


Below is the last few lines of the tracefile with connection lost at 13:50.

2010-09-06 13:10:06:077 (IB) IB.HistoricalDataRequest.Send: last=05/09/2010 instrument='$EURUSD' from='2010-09-04' to='2010-09-05' period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2010-09-06 13:10:06:077 (IB) IB.HistoricalDataRequest.Send: request='EUR;CASH;;0;;;IDEALPRO;;USD;;20100906 00:00:00;1 day;11 M;0;BID'
2010-09-06 13:10:06:077 (IB) IB.HistoricalDataRequest.Send: waiting 14 seconds
2010-09-06 13:10:20:858 (IB) IB.HistoricalDataRequest.Process: itemCount=233 instrument='$EURUSD' from='2010-09-04' to='2010-09-05' period=Daily splitAdjusted=False dividendAdjusted=False bars=0
2010-09-06 13:12:52:882 (IB) IB.Adapter.HandleMessage DataFarmConnectionOk
2010-09-06 13:12:57:835 (IB) IB.Adapter.HandleMessage DataFarmConnectionOk
2010-09-06 13:50:07:881 (IB) IB.Adapter.HandleMessage DataFarmConnectionBroken
2010-09-06 13:50:07:881 (IB) Cbi.Connection.ProcessEventArgs.ConnectionStatusEv entArgs Connected ConnectionLost NoError Market data farm connection is broken:usfuture(2103)
2010-09-06 13:50:07:881 (IB) Cbi.ConnectionStatusEventArgs.Process: Connected ConnectionLost


Thank you...Lee

[NinjaTrader_Jason]

Hello Lee,

Can you please tell me if the freeze up occurs each time you experience a disconnection?

What happens if TWS is installed on the same PC as NinjaTrader and you experience a disconnection - does the freeze up occur as well?

In addition, could you please clarify what type of network is used so we can test it on our end.

[[email protected]]

Hallo.

I also experience this / same kind of / Problem when switching in a chart window using an instrument list from one instrument to another. This can also be viewn when linking e.g a market analyzer with some charts and giving focus to another row in the analyzer. Ninja then has a high cpu load on one thread. The chart window stays whith messaging "Loading data..." The control center window does not have a message that any data is loaded. The thread being in panic never stops. Even closing the window does not help. I usually close ninja and reopen - fine (for some time)...

I am using Ninjatrader 7.0.0.23 64 Bit. The Problem is the same on 32 Bit.

PS.:
Questions:
(1.) Why does ninja reload instrument data that it already has when first time running in the strategy analyzer? When testing the strategy on an instrument list. IB may panic because of the amount of data being requested. Can this handling be twaked by unchecking "Tools|Options|Data -> "Get data from server (if available)" ???

(2) Can the "static" calculation of commissions using levels/tiers be tweaked by proving a user impleneted derivation of a class / implementing some interface. That class would be asked to deliver the comission!

Greetings from Germany

Alexander

[NinjaTrader_Jason]

Hello Alexander,

Do you experience this issue each time you switch instruments in charts. If not, do you see any consistencies between the occurrences so far? When I test this on my end, I do not experience the behavior you describe.

Please see the link below under 'When does NInjaTrader load historical data?' for information how NinjaTrader loads data.

http://www.ninjatrader.com/support/helpGuides/nt7/historical_data.htm

If you uncheck that option, you will not load historical data at all from your connectivity provider.

I will forward your suggestion regarding commissions to Development to put it on the list of possible future enhancements.

[CSharpCoder]

Tx for you answer. I can quite easily reproduce this behaviour. I just have to switch the instrument a couple of times - but its not deterministic how often i have to do this. As a developer i would guess that the handler is not reentrant - the GUI thread within the selectionchange handler doesnt lock the ui of the combo box while still executing the former task. maybe thats why the occurance is more often when the instrument changes quickly and even much more often when connected to IB and therfore awaiting to retrieve data from IB.

This issue is very annoying for me - please help!

[NinjaTrader_Kyle]

CSharpCoder,

Can you provide your workspace and a step-by-step guide that I can use to reproduce the behavior on my end?

Workspaces can be found in the Documents > NinjaTrader 7 > Workspaces folder.

[CSharpCoder]

I found a solution to that freeze problem after instrument change on chart for my installation. The problem was that the cache within the db directory may have been corrupt. I have seen that there were more than 40000+ files with a length of zero. I closed ninja and deleted the whole directory under C:\Users\UserName\Documents\NinjaTrader 7\db\cache\ - that was the remedy for me. Be careful not to delete the precious data - just the cache.

I did not see any freeze since that.

Thanks for the support.

[NinjaTrader_Jason]

Hello CSharpCoder,

Good to hear the freezes do not occur anymore and thank you for posting your solution.

[Lee123]

Hi,
I have NT7v23 and IB installed on computer 1.
I have NT7v23 on computer 2 networked to comp 1 via a SparkCom via a 4 port ADSL router.
Briefly at the same time both NT7 will freeze and IB will show no data (orange).
This can occur when loading data after initial connection or at any time. Whether I am doing something to the charts or doing absolutely nothing.
The trace files only say - 2010-11-29 14:53:01:527 (IB) Cbi.Connection.ProcessEventArgs.ConnectionStatusEv entArgs Connected ConnectionLost NoError Market data farm connection is broken:usfuture(2103)
and further words to that effect.
I have set IB API to error reporting but I cannot find the file or its not making one.
I have trying to get evidence to send you without sucess, however I am now of the opinion that it is not IB at fault but either the router or NT7 on computer 2.(networked).
NT7 on comp 1 will run perfectly and reconnect to IB perfectly as long as NT7 on comp 2 is not running.
NT7 on comp 2 will freeze by itself when NT7 on comp 1 is not running.
So my conclusion is that it is NT7 networked on computer 2 that causes the problem.
I cleared the router log at the beginning of the session and have been checking it to see if it remained clear. It did remain clear until the freeze when it recorded the following:


Is it possible for NT7 to knock out a router.
I hope the above is of use because this is happening almost on a daily basis and sometimes more the once.

Thanks...Lee

[NinjaTrader_Jason]

Hello Lee123,

If the lower left corner of the Control Center reflects an orange color, it indicates you are not properly connected anymore.

Do you get re-connected on computer #2 after the Control Center displays an orange color? Please note that NinjaTrader will try to get re-connected for about 2 minutes. If no connection could be established, NinjaTrader will stop trying. Could it be the connection never restores and thus you do not receive any real-time data anymore?

Am I correct TWS on computer #1 feeds NinjaTrader on computer #2 with data?

Unfortunately I am not sure how to interpret the router information.

[Lee123]

Hello Jason,
thanks for the reply.
Yes IB on computer 1 feeds computer 2.
Sorry I need to clarify: NT7 control center if it is visible will be red. IB will be orange.
There will be no action on the charts, When I click on any part of NT7 the mouse icon gives the busy signal and NT7 freeze. Sometimes when I close NT7 via the windows task manager (as I have to do) a window will show which says"sorry NT7 will have to close"
There is no reconnection by NT7 in this situation.
I do not understand router either, but I can see that its session has been lost and it is reconnecting. If I manually switch off the router and then switch back on then both NT7 will go to red and will not freeze, they only need to be reconnected.
Now I don't know which is initiating the problem, either NT7 (networked) or the router. I am inclined to believe NT7 because I have not had these problems before and a number of freezes happen at the precise moment that I click on a chart for example (to many coincidences). However if it is the router malfuntioning then surely NT7 should just go red and not freeze.

Thanks,
Lee

[NinjaTrader_Brett]

Lee,

Not necessarily, as an intermittent router issue I could see this occurring. Also, a router power off and power on in our developers eyes is not a proper test scenario for disconnect and reconnect of NinjaTrader as this does not have the characteristics of a normal internet interruption.

I suggest to you to simplify your setup by taking the router out momentarily to see if the same issue occurs.

Let me know if I can be of further assistance.