Performance question about lookback period

[JS999]

This is a fairly technical question, that might need input from NT's development staff.

I know that if you increase the look-back period on a chart, it decreases performance... for example, if you set a chart to look back only 200 bars, it should result in less CPU usage than setting a chart to look back 100,000 bars. (We're talking about a 3 minute chart here, just as an example.)

The question is, why does this decrease performance? My theory is that if you have indicators on the chart, the code will re-calculate that indicator across the entire length of the look-back series. Particularly if you have it set to CalcOnBarClose = false so that it re-calculates every tick, I could see how this would be a problem with a lot more bars.

However, does the same performance issue apply if you have no indicators on the chart? Or in a third scenario, what if you have indicators, but they are always set to calculate on bar close = true?

I guess I am trying to figure out if the code bottleneck that causes the length of the data series to affect performance is in the indicator calculation part of the code, or the actual screen drawing part of the code. I can't see how it should be any slower if you have a chart with 100,000 bars on it, if you are only showing the last 100 bars and you either have no indicators or have every indicator set to relcalculate only at the close of each bar, which would be every few minutes. (I would hope that it's not somehow "drawing" all of the 100,000 bars somewhere if they are not shown on screen, for example.)

Any input from NT's dev staff would be appreciated here, since this is probably a fairly technical question... thanks!

[NinjaTrader_AdamP]

JS999,

I just loaded 10 charts with 444 days of 1 minute bars, which is about 639360 bars each assuming I have access to historical data back this far, and I am having no chart slow down when scrolling, navigating, placing simulated trades, etc. My PC has an Intel Core i5 dual core processor in it, and 4 GB of RAM. Most people have something comparable or much better. The only thing that took longer than a few milliseconds was actually downloading the historical information itself so I could generate the charts. This being the case, I suspect that the issue is elsewhere.

The next step I did, was add my custom template that has 6 different indicators (NinjaTrader defaults) on it. It took awhile to load, since the indicators are calculated on 639260 bars, but after it finished I could navigate around, place simulated orders, etc. on all 10 charts without any slow-downs. I then set CalculateOnBar=false and also experienced no slow downs.

Indicators for very long data series will only cause a delay initially when they are added to a chart because it has to calculate indicator values for all the bars. This is to be expected, as computers have a maximum number of calculations they can do per second, and can only move data around in memory so fast based on various hardware specifications. If an indicator has to do 20 additions and 1 division to get its value (like an SMA), that is 21 operations and must do so on a 639360 element array, that is 13,426,560 calculations, while also having to move data around and store it. Note : I didn't exactly follow CS theory here for computational complexity, but its an estimate.

Indicators that come with NinjaTrader in a new installation (except Swing) only calculate once for each bar. Even if CalculateOnBarClose = false, it will only update the most current bar tick-by-tick and leave all the other prior bars alone. As long as indicators are designed similar to the NinjaTrader default indicators (i.e. the ones that come with a fresh install of NT), it won't usually cause performance issues except for the initial delay when calculating on extremely long data series.

If you are using custom indicators, sometimes they will go back and access numerous prior values, or sort arrays, etc. When CalculateOnBarClose = false for these, and sometimes even with CalculateOnBarClose = true, it can cause performance issues. Some poorly designed custom indicators will access and change previous values. This is known as "repainting" and is usually frowned upon. Essentially, I would look closer at the custom indicators you choose to use and ensure that they aren't doing one of the aforementioned things.

Please let me know if I can assist further.

[JS999]

Thanks for the response, Adam... I guess I was under the impression that more bars caused a slowdown, based on conversations I had with NT reps a few years ago with NT 6.5. It's good to hear that longer data series have no real effect on performance once they are loaded, which is the way I expect it should be working. Thanks again!

[NinjaTrader_AdamP]

Js999,

No problem.

Please make sure you are using the latest version here : http://www.ninjatrader.com/download-registration.php

Have a nice day!

[JS999]

Actually, one more question did occur to me... not that I think it should make any difference, but what if you had a 1 minute chart with all of those bars loaded, and with Global drawing objects set to True. Now imagine another chart, say an hourly chart, also looking back many bars.

Now you draw a global trendline on the hourly chart and it shows up on the 1 minute chart as well, which is the expected behavior. Would having this very long trendline on the 1 minute chart stretching back across thousands and thousands of bars impact system redraw performance in any way? That is the reason that I would have a chart with a lookback period that long, so I wonder how the drawing algorithm works, and if this would have any effect on it... do you have information on that by any chance?

Thanks again!

[NinjaTrader_AdamP]

JS999,

No, global drawing objects should not affect performance on long data series as long as the drawing objects aren't being calculated based on a large number of elements of the data series every tick.

Basically, if you draw something yourself using a drawing tool, even if its a global drawing object, it shouldn't affect performance.

Please let me know if I can assist further.

[JS999]

JS999,

No, global drawing objects should not affect performance on long data series as long as the drawing objects aren't being calculated based on a large number of elements of the data series every tick.

Basically, if you draw something yourself using a drawing tool, even if its a global drawing object, it shouldn't affect performance.

Please let me know if I can assist further.

Yes, I was referring to manually drawn objects and not those drawn by an indicator. I do actually also have objects that are drawn by an indicator (vertical and horizontal lines for example), but those are being drawn on the close of every bar, so I don't expect that should be an issue. Thanks again!