Why loading indicator take so much time when AddChartIndicator used?

[GaryAlbers]

I have a strategy that loads several indicators via AddChartIndicator.
This scenario uses 'TickReplay = true' and one of the indicators and the strategy are set to 'CalculateOnEachTick'.

The overall load time of the strategy is SIGNIFICANTLY slower than the sum of the load times for the individual indicators and the OnBarUpdate time of the strategy.

In trying to isolate where the time is being spent I determined that it is in the Update() method for the loaded indicators.
The issue is that the amount of time is disproportionately bigger when the strategy loads the indicators versus just loading the same indicators on the chart.

In my test (about 690,000 calls to OnBarUpdate) the total strategy load time is 80 seconds. If I load those same indicators alone on the chart the total load time is about 10 seconds.

I am pretty sure the time is spent in the call to Update() because I tried 2 versions of the strategy: 1) The strategy calls the Update method on each indicator in OnBarUpdate and 2) The strategy does not call Update , leaving it up to Ninja to call it, which it does.
I could measure the time that the calls to Update took and it was about the same in both versions, except that in version 1 the time was in my code and in version 2 the time was in Ninja code (outside my OnBarUpdate).

The net result was about the same total load time but when my code called Update(), about 60 seconds of the time was in the strategy OnBarUpdate method. When strategy did not call Update, then about 78 of the 80 seconds was spent outside of OnBarUpdate.

This seems fishy since when loaded together on a chart the worst case is about 10 seconds. Something is surely inefficient when it takes Ninja 70 seconds longer to process the same indicators when loaded from a strategy.

Any ideas on what is causing the huge difference in load times?

Thanks,
Gary

[sledge]

This seems fishy since when loaded together on a chart the worst case is about 10 seconds. Something is surely inefficient when it takes Ninja 70 seconds longer to process the same indicators when loaded from a strategy.

Any ideas on what is causing the huge difference in load times?

Is your CPU maxed out at 100% in Task Manager?

[GaryAlbers]

CPU load - nothing significant

I took two screen shots of TaskManager:
1) while Strategy was loading (cpu at 12% total)
2) after it was done (cpu at 12% total)

Note that the data feed is live and there is a MarketAnalyzer running with 12 instruments updating so Ninja is doing stuff but nothing significant.

So the answer is no, the CPU is not pegged.


Gary

[NinjaTrader_PatrickH]

Hello GaryAlbers,

Thank you for your report on this matter.

We will test this on our end. Any information we find we will report here.

[NinjaTrader_Jesse]

Hello,

I wanted to reply back on this item,

Currently I have only been testing using standard items included with the platform but have been unable to see the same outcome.

Based on the initial inquiry, this seems this could be caused by one of the items being loaded by the strategy. Can you tell me, do you currently only have standard indicators loaded or are there other third party items being used?

If you are able to demonstrate this using only standard items, I would like to get a copy of the strategy to further test. Otherwise if you have not yet tried this, it would be an easy way to determine what the general cause of the slowness is to further address that item specifically. Creating a sample that utilizes only included indicators like the SMA for example for testing can help eliminate possible causes.

I look forward to being of further assistance.

[GaryAlbers]

This begins to reproduce it

I created a strategy and an indicator that require every tick.
The strategy also calls AddChartIndicator on a bunch of standard indicators.

In my environment the strategy takes 22-23 seconds to load.

If I load a chart with the exact same indicators, the chart loads in 6-8 seconds.

Something is up that the strategy takes so much longer.

As my original post shows, in my real code, the difference is much bigger and grows the more bars are included in the chart.

So I guess the real question is what accounts for the HUGE difference in load time for the strategy that does nothing but load indicators versus a chart with the same indicators?

Thanks for looking into this,
Gary

[NinjaTrader_Jesse]

Hello,

Thank you for the sample, I will review this further.

I did have one question after looking the syntax over, I wanted to question the use of Update() on all the indicators supplied.

Are you calling Update on all indicators in the script at all times or on every OnBarUpdate? or is update being utilized in the indicators public property that needs Updated as demonstrated in the help guide? http://ninjatrader.com/support/helpG...ightsub=Update

Based on every indicator being force updated I could see there would be a performance impact, I would need to verify what the reason for that would be but it would also help knowing the specific reason for using Update in this way.

I look forward to being of further assistance.

[GaryAlbers]

In the past, using NT7, I ran into a number of instances where it was necessary to call the update method on an indicator that was NOT added to the strategy (via Add) but was just used internally. Also, I was comfortable doing so because the support comments I have read plus my experience doing it has shown that it was not a performance problem. so I started doing as a habit.

When this performance problem became apparent, I tried both ways.
I am not seeing a difference in the total elapsed time. I ask you to try the submitted code also. Run two tests, one with the calls in and one with the calls commented out.
Look at the output window after each test and I believe you will see the total elapsed time is pretty close.

So now the question becomes: why is there so much additional time to load the strategy versus loading a chart with the same indicators on it?

If I can load a chart with a bunch of indicators and it takes 6-8 seconds to load, then for a simple strategy that does not do anything, I would expect that strategy to also load in about 6-8 seconds, or maybe a bit more. However, in my tests on my machine, I am seeing 22+ seconds.

Gary

[NinjaTrader_Jesse]

Hello,

I just wanted to reply to let you know I have submitted this item for further review.

If I have any further comments or developments I will reply back with that information.

I look forward to being of further assistance.

[GaryAlbers]

An update

Using beta 11, I re-ran the strategy startup test using the attached code.
Once again the output shows that the bulk of the time is spent outside my code, with the likely explanation being in the strategy loading process (Update() method).

Here is the output from my test:

GTestLoadTickIntensiveIndicators: From Historical to Realtime state (seconds) = 96
GTestLoadTickIntensiveIndicators: OnBarUpdateTime(seconds)=1
GTestLoadTickIntensiveIndicators: OnBarUpdateCount=1,610,464
GTestTickIndicator: From Historical to Realtime state (seconds) = 96
GTestTickIndicator: OnBarUpdateCount=1,610,464
GTestTickIndicator: tickCounter=1,610,464

There is one indicator that listens to every tick, and here is it's
load time when it is the only indicator on the chart (no strategy):

GTestTickIndicator: From Historical to Realtime state (seconds) = 4
GTestTickIndicator: OnBarUpdateCount=1,610,464
GTestTickIndicator: tickCounter=1,610,464

If I comment out the TickIntensive indicator here is the strategy load time:

GTestLoadTickIntensiveIndicators: From Historical to Realtime state (seconds) = 85
GTestLoadTickIntensiveIndicators: OnBarUpdateTime(seconds)=1
GTestLoadTickIntensiveIndicators: OnBarUpdateCount=1,610,464

When the strategy loads the tick intensive indicator only and the other 11 indictors are on the chart directly, here is the load time:

GTestLoadTickIntensiveIndicators: From Historical to Realtime state (seconds) = 10
GTestLoadTickIntensiveIndicators: OnBarUpdateTime(seconds)=1
GTestLoadTickIntensiveIndicators: OnBarUpdateCount=1,610,464
GTestTickIndicator: From Historical to Realtime state (seconds) = 10
GTestTickIndicator: OnBarUpdateCount=1,610,464
GTestTickIndicator: tickCounter=1,610,464

So, it apears that the tick intensive indicator is not the culprit.

When all 12 indicators are loaded on the chart directly (no strategy) I had to meaure it visually, but it was roughly 25 seconds.

I think there is sufficient evidence that there is something very slow about how the strategy mechanism is loading and processing historical data with lots of indicators. In my previous posts, I played with calling the Update method in the strategy, which I no longer do, but it did show that the Update method appeared to be where the bulk of the time was being spent, whether I called it or Ninja called it.

In real time, on a product with lots of ticks, I experience over 2 minutes of load time on a chart. This is not good.

I would appreciate any feedback or update on this issue.

Thanks,
Gary

[NinjaTrader_MichaelM]

Hello Gary,

The Calculate behavior (ex: Calculate.OnEachTick) of the hosting strategy determines the calculate behavior of the hosted indicators unless the hosted indicator overrides the Calculate setting in State.Historical.

Scripts that require Calculate to be set by the developer must set this property in State.Historical in order to ensure that if this script is a child (hosted) that the parent.Calculate property which is adopted by the child is overridden again.

REF: http://ninjatrader.com/support/helpG...ightsub=hosted

I have attached a test strategy/indicator to illustrate. The indicator is set to Calculate.OnBarClose and the strategy is set to Calculate.OnEachTick. You will see in the output that the indicator ends up calculating on each tick. For this reason, you would want to load all of your indicators in your chart test with Calculate.OnEachTick, or modify the indicators as above.

This should account for some of the time difference in your tests, but I am still seeing some discrepancies which I am investigating further.

Thank you for your continued patience in the meantime.

[GaryAlbers]

Michael,

Thanks for the tip to have the hosting strategy set the Calculate flag of each indicator when changing to Historical state. I updated my test code and my real strategy to do this and the performance improvement was negligible. I attached the updated test indicator.

I am glad to have the code more correct but I am really hoping there is more that can be done. It just makes no sense that the strategy loading takes so long when only a fraction of the time is actually spent in my code. It does not appear to be a direct result of calculating on every tick either.

Thanks,
Gary

[sledge]

Michael,

Thanks for the tip to have the hosting strategy set the Calculate flag of each indicator when changing to Historical state. I updated my test code and my real strategy to do this and the performance improvement was negligible. I attached the updated test indicator.

I am glad to have the code more correct but I am really hoping there is more that can be done. It just makes no sense that the strategy loading takes so long when only a fraction of the time is actually spent in my code. It does not appear to be a direct result of calculating on every tick either.

Thanks,
Gary

I connected to Playback-Market Replay - which autoplayed 1 day of 15 seconds chart of ES 06-16.
I added your indicator/strategy, and it instantly appeared.

I didn't see what type of chart you are applying this too - and how many days you have loaded.

------------------
Update: I did a replay of ES 03-14 from 12/14/2016 through 2/26/2016 of a 1 second chart. I added your strategy, and then enabled it. It took about 30 seconds (I counted in my head).

[GaryAlbers]

I am using range bar charts on CL and ES with TickReplay on. Two of the 10-12 indicators that the strategy loads depend on the TickReplay.
If I load 1-3 days of CL, it can pull in 1.5 million ticks.

However, I have timers in all the code where I process the ticks. As the statistics that I posted show, the bulk of the time is not spent in my code.
It appears to me to be when Ninja calls Update() for each indicator.

Gary

[NinjaTrader_Matthew]

Michael,

Thanks for the tip to have the hosting strategy set the Calculate flag of each indicator when changing to Historical state. I updated my test code and my real strategy to do this and the performance improvement was negligible. I attached the updated test indicator.

I am glad to have the code more correct but I am really hoping there is more that can be done. It just makes no sense that the strategy loading takes so long when only a fraction of the time is actually spent in my code. It does not appear to be a direct result of calculating on every tick either.

Thanks,
Gary

Hi Gary.

We did run your scenario by our lead engineer. As simply said as possible, there is an extra burden added to the core processing in order to host an indicator versus just adding it on a chart. These two scenarios are not apples to apples. He did not see much that can be done at this time to improve the performance other than what has been remarked in this thread.

We are keeping an eye out for unusual behavior in this area to see if there is something which can be uncovered that may warrant additional research, which could ultimately improve the performance hereabouts. However, there is nothing we have identified immediately that would help at the time of this post.