CPU Usage with IB Data Feed

[aviat72]

I am observing a peculiar behavior with the IB data feed.

1 Even if I have one workspace active with just a single chart open NT quickly hogs 25% of the CPU (single core on the quad core machine). (See NTCPUSingleChart.png)

2 If I have a large workspace with many charts, even then the CPU usage stays at 25%. (see NTCPUDualWS). I also have an empty WS open at the same time.

3 If I switch to the empty workspace while keeping the WS with a large number of charts active in the background, the CPU usage literally drops to zero. (NTNoUsageDualWS)

I have made sure that the indicators are all set to CalculateOnBarClose = true. All the charts are 1 min charts. I have shut down data-storage on disk by NT (no replay no save historical data etc.). The resource manager shows literally no disk traffic from NT.
-----------------------------------------------------------
Some Notes:

(a) What is peculiar is that NT takes 25% of the CPU irrespective of whether it has one chart or 12 charts. Also the CPU usage collapses if no chart window is open in the active workspace even if a large workspace is open in the background.

(b) Notice how there are 24 threads open in the case with the single chart with NT. It seems there is some thread which is hogging the CPU and that situation occurs when a chart is Open and being plotted in a live workspace.

(c) Even if you increase the number of charts, the machine does not die, indicating that the thread is in some form of a wait state but still consumes a lot of cycles. As the real work increases with more tasks, the individual threads take less time but the CPU usage is still 25%.

The problem here arises when you want to make a change (i.e. open a chart). Then the dialog boxes die and do not work. Perhaps the threads for these have a lower priority and hence never get activated by the scheduler. I have to kill NT via the task manager and restart.

This was with Beta13.

[NinjaTrader_Kyle]

Hello aviat72,

Thank you for your post.

Please help us to further isolate this issue by removing all indicators from your charts. After doing so, please let us know if the CPU amount remains the same.

[aviat72]

Kyle:

I am going to try with NTBeta14 first.

The first signs look promising though nothing will be certain till the market opens tomorrow and we get a lot of market data coming in.

[aviat72]

This problem is related to NT not being able to handle the quirks of the IB datafeed gracefully. There is also data-corruption where the ES data was somehow stored in the 6E database.
__________________________________________________ _________
I continue to have problems with IB Data Feed and NT7B14 where the NT takes 25% of the CPU time. It is happening even now after market hours.

I think the problem is related to failed data-download requests from IB.

The log has the following messages:

4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|3|4|Requested market data is not subscribed. 354
4/29/2010 8:16:36 PM|0|4|Error on loading chart data for '6E 06-10 Globex': IB: Error in requesting historial data from IB: No security definition has been found for the request(200)
4/29/2010 8:16:37 PM|0|4|Error on loading chart data for 'NQ 06-10 Globex': IB: Error in requesting historial data from IB: No security definition has been found for the request(200)

I am not sure whether this is due to a pacing violation or some other issue but after this happens, NT continues to remain in the 25% CPU usage mode and almost unresponsive.

=> I was able to disconnect from the IB Data Feed (it took minutes for the mouse response) but the CPU usage continues to be at 25%.


=> I had to kill the process above. I restarted NT and when I reopened the workspace it hung NT again. The culprit seems to be the corrupted 6E database. I am going to delete the 6E data (see below) for today manually next.

=> Deleting the database entries and then reconnecting to IB seems to have fixed the problem. It downloaded the 6E data for today from IB and is working fine now.
------------------------------------------------------------------
A bit of background:

Prior to connecting to IB, I had downloaded 1 month of tick accurate data from IQFeed and none of the charts in the active workspace go back more than a month. I did that because the data-dbs were somehow corrupted to the point where opening a workspace without an open data-connection would also hang NT.

After downloading the data I reopened the workspace which then opened fine. I then shut down the IQFeed connection, and opened a connection to IB and after the attempt to reload the historical data, NT never recovered. It responds to a click after a minute or two.

I am attempting to change the options so that NT does not try to download historical data from the server.

If it helps, currently the PnF chart for 6E is showing a PnF chart for ES (the last price being 1202.50!!) (see attached picture).

[NinjaTrader_JoshP]

aviat72,

Could you check the Instrument Manager for your 6E and what its symbol mappings are?

Do you receive any error messages? Popup or in the Control Center log tab?

[Harry]

NT 7.0.0.15 unusable

NT 7.0.0.15

There is a new CPU usage problem with IB data feed probably caused by a bug fix for NT 7.0.0.15

NT 6.5. / NT 7.0.0.14 with 6 charts and with many indicators set to False for CalculateOnBarClose -> takes below 5 % of my CPU

NT 7.0.0.15 with one chart with all indicators set to True for CalculateOnBarClose -> takes between 15% and 20% of CPU

Cannot use NT 7.0.0.15 in any way even with simple indicators. CPU heats up and ventilator goes to max.

Is there any link so I can reinstall NT 7.0.0.14 please?

[NinjaTrader_Kyle]

Hello Harry,

Thank you for your post.

Please send a note to Support [AT] NinjaTrader [DOT] com with 'ATTN: Kyle' in the subject line.

In the message, please include the following:

1.) A link to this forum thread.

2.) Your most current trace file.
You will find the trace file in the My Documents > NInjaTrader 7 > Trace folder.
The file will be named 'trace.YYYYMMDD.txt'

[Harry]

Thanks Kyle, will send it within five minutes.

Screenshot left NT 6.5. (workspace 5 charts): CPU usage 1.5%
Screenshot right NT 7.0.0.15 (workspace 3 charts) : CPU usage 17%

The workspaces have similar indicators.

[aviat72]

Josh the product mappings are all fine and default. It was working fine before the IB violation happened. I have noted the error messages in the log extract above. These messages also popped up.

After the violation happened (the log messages above) it saved bad data. i.e. the data coming for ES was saved in 6E. I manually deleted the last day's data for 6E and then reloaded and it downloaded the correct data from IB.

I have downloaded NT7B15 and even that is showing 25% CPU usage. However NT is still responsive to mouse clicks though it is slow. I have checked with the resource manager that the no other resource (network, hard disk) is the bottleneck. It is the CPU usage which is the problem.

Some key observations:
-The CPU usage is independent of the number of charts it seems. This suggest that there is a thread which is polling or doing repeated wasteful tasks when linked with IB. As more charts come on line the thread scheduler reduces the time assigned to the wasteful threads but they still keep the CPU running at full 100%.

-The interface with IB is not handling IB's quirks gracefully. I got a message that IB did not recognize 6E as a valid symbol but then NT put the data coming in for ES into 6E.

I wish I could talk to a developer and help them debug this. Right now I have to speculate on what potentially could be happening under the hood based on what I see. Since the behavior is intermittent it can be hard to reproduce.

But Houston, we have a problem.

[Harry]

Confirm all findings of aviat72

Can confirm all findings below. You can check the screenshots below taken from the process explorer, showing the difference between NT 6.5. and NT 7.0.0.15. With modest charts my CPU is running at 17% compared to about 2% before. Problem occurs already with a single 5 min chart. I usually only have my bar colour indicator and volume set to False for CalcOnBarClose, and these are very simple indicators, so it is data feed related.

I have downloaded NT7B15 and even that is showing 25% CPU usage. However NT is still responsive to mouse clicks though it is slow. I have checked with the resource manager that the no other resource (network, hard disk) is the bottleneck. It is the CPU usage which is the problem.

Some key observations:
-The CPU usage is independent of the number of charts it seems. This suggest that there is a thread which is polling or doing repeated wasteful tasks when linked with IB. As more charts come on line the thread scheduler reduces the time assigned to the wasteful threads but they still keep the CPU running at full 100%.

[NinjaTrader_Kyle]

Hello Harry and aviat72,

Since we are in contact with both of you via email, we will follow up there.
Thank you for reporting this.

[aviat72]

An update:

I downloaded NT7B15.

The computer ran at full CPU speed and 25% (100% of a core) all through out the market day. The charts were updating and NT was responsive though slow.

After the market closed at 4:15PM EST the CPU usage has gone down to 0%. This is tied to how NT interfaces with the IB data-feed.

[NinjaTrader_Kyle]

Hello aviat72,

Please provide me with the workspace you were using today via the email thread we have going already.
Thank you!

[Harry]

What is happening?

CPU is really a huge problem. I have a simple workspace with a 1 min chart, a 5 min chart and a 30 min chart. With the exception of the coloured bars and volume, all indicators are running with CalcOnBarClose = true.

I do not know what is happening here, but with this NT 6.5. and up to NT 7.0.0.14 never used more than 5% of CPU usage (peak). Since Friday it is firing up to 40%. See chart below, TWS uses 0.77% NT 36.92% of CPU.

I would really love to get NT 7.0.0.14 back to test it, I think this problem is a new NT 7.0.0.15 bug.

Have not yet built a testing workspace, but anyhow NT 7.0.0.15 is close to unusable.

[NinjaTrader_Kyle]

Hi Harry,

To further investigate, I will need you to send a note to Support [AT] NinjaTrader [DOT] com with 'ATTN: Kyle' in the subject line.

In the message, please include the following:

1.) A link to this thread.

2.) The workspace you are using (My Documents > NinjaTrader 7 > Workspaces).

3.) Any indicators that are applied to the workspace.
These indicators can be exported via the File > Utilities > Export NinjaScript menu.