CPU Usage with IB Data Feed

[Harry]

Cannot duplicate the workspace now, too many proprietary indicators. Problem is somehow linked to custom plotting.

When an indicator uses a custom plot, even when for this indicator CalculateOnBarClose is set to True, the plot is executed with each incoming tick and slows down NinjaTrader. Covering the plot with another application solves the problem, but you do not see the charts any more.

[NinjaTrader_Kyle]

Hi Harry,

If you create a similar workspace but use only default NT indicators and the issue does not occur, this would indicate that the issue itself has to do with those custom indicators and how they interact with NinjaTrader.

[Harry]

Test Workspace built in a few minutes

So I have just built a workspace with system indicators, which shows similar behavior. On my machine it already takes between 7% and 22% of CPU, which is ways too much.

It would be interesting to take this workspace and make a CPU test with NT 6.5., NT 7.0.0.14 and NT 7.0.0.15 - I will do this on my side for NT 6.5. and NT 7.0.0.15

The workspace has three charts, 1 min, 5 min and 30 min (no tick charts!) with a lookback period of 65 days each. All three charts use identical chart templates (for simplicity reasons) and the three charts, DOM and control center are displayed via 2 monitors. The charts have pivots, but these are on purpose set to CalcFromIntradayData not to create any supplementary difficulties with the separate system threading.

I attach a screenshot of the workspace and the zipped file of the template.

Hi Harry,

If you create a similar workspace but use only default NT indicators and the issue does not occur, this would indicate that the issue itself has to do with those custom indicators and how they interact with NinjaTrader.

[Harry]

4.5 times more CPU usage NT 7.0.0.15 versus NT 6.5.

Sorry previous workspace contained Klinger oscillator, forgot that this is not a system indicator, so I replaced it with a simple MACD. Below you find identical templates for NT 6.5 and NT 7.0

I tested both of them for CPU usage:

NT 6.5. between 1.56% (min) and 4.69% (max)
NT 7.0.0.15 between 7.03% (min) and 22.66% (max)

This means that NT 7.0.0.15 needs about 4.5 times more CPU usage to display the same workspace!

Hope this explains what I mean!

[NinjaTrader_Kyle]

Hi Harry,

Thank you for providing these files!

I just saw your email come in. I will test on my end and follow up with you there.

[Harry]

My workspace behaves actually worse than this one, but that's the best I could do within a few minutes. Just allows to show that NT 7.0.0.15 has a CPU problem. Would really love to switch back to NT 7.0.0.14, because I think the problem was introduced with v15!

Hi Harry,

Thank you for providing these files!

I just saw your email come in. I will test on my end and follow up with you there.

[aviat72]

Harry
I had the same problem with V14. It behaves a bit better with V15.

[Harry]

My problem is with V15, but I haven't made any CPU test for V14.

Harry
I had the same problem with V14. It behaves a bit better with V15.

[Harry]

Help please, NT 7.0.0.15 is unusable

I have a huge problem with NT 7.0.0.15

With my simple 3 chart default workspace it takes 40% of CPU all the time. The fan is running at max. speed. Something is terribly wrong.

Could I go back to NT 7.0.0.14, please?

[NinjaTrader_Kyle]

Hello Harry,

We are still in the midst of investigation on this issue.

In our tests, the workspace is performing at the same level in terms of CPU usage in versions 7.0.0.15 and 7.0.0.14, so unfortunately, rolling back would not help in this case.

We will update this thread with more information after completing our investigation. Thank you for your patience.

[Harry]

Hi Kyle,

thanks for the information. It is well possible that NT 7.0.0.14 behaved in a similar way, and that I just did not notice it..... But I do not think that all prior versions of NT 7.0 have shown this behavior.

Hello Harry,

We are still in the midst of investigation on this issue.

In our tests, the workspace is performing at the same level in terms of CPU usage in versions 7.0.0.15 and 7.0.0.14, so unfortunately, rolling back would not help in this case.

We will update this thread with more information after completing our investigation. Thank you for your patience.

[NinjaTrader_JoshP]

Harry,

Please find attached a custom Pivots indicator which should address the performance issues. Please install this version and overwrite the old one and see if you still have performance issues with this version. On my end, the workspace you attached earlier which use to take ~20-30% CPU now takes ~5-10%, in line with NT6.5's results.

Should you see any inaccuracies to the pivots we will look into it promptly as well. Thank you for your efforts.

[Harry]

Hi Josh,

have tested the pivot indicator. It definitely is an improvement over the prior version. I measured the CPU usage with the new indicator, and for the offending workspace it is still about 80% higher than for NT 6.5.. But with the prior version of the pivot indicator I had measured a value of about 350%, so for me this solves the issue.

My suspicion is that the CPU usage is caused by the custom plot and that part of the code was therefore transferred away to a method which would only be called on bar close. Could you confirm this, as this would be important to know for the coding of other indicators.

Otherwise the pivots indicator works well. There is just a minor point to note: The indicator also calculates and displays values if the reference period was half a week or half a month. It should be modified to display nothing instead of false values, when the first period (week, month) is incomplete.

[NinjaTrader_JoshP]

Harry,

When you say 80% higher, what are the exact values you are seeing? I see 5-10% CPU on NT6.5 and 7 on my end with the exact workspaces you attached earlier.

There were calculations being done in the Plot method of the Pivots indicator that was expensive performance wise and it was being done multiple times. So, yes, it was moved away to be less intensive. Development says it is unlikely you would run into the same issue with your own programmings.

Thank you for the suggestion on half week/months.

[Harry]

With NT 7.0 CPU load was between 3 and 9%, with NT 6.5. CPU load was between 2 and 5%, when I tested, test performed in the morning with slightly lower tick rates. Of course the result depends on the processor used.

As you know, I coded some session pivots that basically rely on the code of the pivot indicators, but display 3 times as many lines. So any CPU problem caused by the original pivots is multiplied by 3. This explains my interest in understanding where the offensive code is located.

Harry,

When you say 80% higher, what are the exact values you are seeing? I see 5-10% CPU on NT6.5 and 7 on my end with the exact workspaces you attached earlier.

There were calculations being done in the Plot method of the Pivots indicator that was expensive performance wise and it was being done multiple times. So, yes, it was moved away to be less intensive. Development says it is unlikely you would run into the same issue with your own programmings.

Thank you for the suggestion on half week/months.