Problems getting chart updates for data

[RandyT]

NT started having problems earlier today with charts freezing. Things got progressively worse through the evening as I tried to work on backtesting. I have one instrument that properly updated out of the 4 I normally maintain.

I reset the DB, cleaned the DB, restarted and continue to have problems. I've cut data lookback times to 10 days and turned off every other data option that I think could cause a performance problem. ie sim data, storing data, etc.

I'd sure like to have this working by US market open....

[RandyT]

BTW I have confirmed that I am getting data from my DTN feed.

[NinjaTrader_Jason]

Please see the link below for more suggestions to reduce the RAM load of NinjaTrader.

http://www.ninjatrader-support.com/HelpGuideV6/PerformanceTips1.html

Make sure you do not have workspaces running in the background. They are listed at File-->Workspaces between "Save Workspace As" and "Default". They will request data and RAM as well.

Reduce the amount of indicators on your chart if you have these applied. Make sure their "Calculate on bar close" option is set to True.

If the issue persists, please reduce the amount of other windows in your workspace.

[RandyT]

I am well ahead of you on these suggestions. This is not a new configuration for me.. it is the same setup I have been trading for a couple of weeks. It has suddenly ground to a halt.

This morning, I have 3 of 4 instrument charts updating, but the 4th is hung with data from the 13th, even though I traded it all day yesterday and had data for the 14th...

[NinjaTrader_Jason]

If you close the 4th chart and open up a new one, does it populate with data?

Try right clicking in the chart and select "Reload Historical Data".

[RandyT]

So far... no.. I tried to force a reload of historical data and that dialog box is still up. I then tried your suggestion of a new chart, but the chart is just blank and the "Loading data" dialog box is still up.

[NinjaTrader_Jason]

I suggest to close this workspace and create a new one. Save the workspace under a different name and check if the issue persists using the new workspace.

[RandyT]

Ok, after the amount of time it has taken to write these two emails, I now have a chart running on that market.

Here is the followup question:
I went back and forth with NT support on a similar issue about 6 months ago before I purchased NT. It makes new sense to me that I can open up 2 charts on the same instrument and EACH chart will take over 2 minutes to load. It makes me question the cache efficiency. I see similar issues when backtesting where NT will reload for every backtest run.

I'm ok with this if it is a recognized problem and we have some hope of improvement with NT7. I am very concerned if this is still an unrecognized problem with no end in sight.

[NinjaTrader_Jason]

Could you please check if the charts use the same interval and if the look back period is the same. If not, NinjaTrader will request the data from your data feed provider.

NinjaTrader will load data using cache/database when you would change intervals of same interval type (tick, minute, daily) in the same chart.

[RandyT]

I've seen the behavior using minute charts in intervals of 1, 5, 8, 15, 30, 60,... I seem to see the same behavior using Volume and Range charts.

I guess it makes no sense to me that if I am getting tick data from my data provider, that Ninja should be doing anything but requesting missing tick data and rendering that tick data through local cache/db reads into whatever interval I request. That does not appear to be the behavior.

I also cannot explain why I can run backtest after backtest of the same instrument and interval and sometimes it will go through a loading stage for the data, and other times it does not.

Sorry to sound whiny, but it seems there is a code problem here. If there is not a code problem, then I think that I am trainable enough to solve the problem through different behavior. The later solution does not seem to be working.

For reference, the past support thread was about 6 months ago with Jessica. I gave up on NT at that time given how long it was taking just to bring up an environment. But.. I am baaack and am a paying customer now. :-)

BTW, this is running on a 2.4ghz Quad core with 8GB of RAM. I don't have a resource problem.

[RandyT]

Let me add another example since it is fresh and in front of me at the moment...

I have been backtesting a strategy on the EMD continuous contract for over 2 weeks now. Same time interval, same range of data (1/1/2008 to today). The first run of the backtest must have taken 15 minutes to run. Just right clicking on the Instrument symbol to rerun the strategy caused the Data Loading dialog to pop up and it did something for about a minute.

Why?

[NinjaTrader_Jason]

During the minute the dialogue box is displayed NinjaTrader is recalculating your backtest results.

Please take a look at the following link, which will provide you more information on how NinjaTrader loads data.

http://www.ninjatrader-support.com/HelpGuideV6/WhenDoesNinjaTraderLoadHistoricalData.html

In addition, I suggest to disconnect from your data feed provider once you downloaded the data, which you will use to backtest on.

[RandyT]

As I dig into this further, I am not getting a full continuous contract data for this symbol. I suspect that NT is spending a lot of time trying to get that data. Knowing what it is doing would help here.

The calculation should not take a minute. I can see that in the backtesting Prints that I am seeing. The backtest process sits "loading data" for about 5 minutes per run, but the calculation takes only a about 1 second.

We are spending time doing things that we should not be doing. If we cannot get the data, it seems some notice of that and returning more quickly would be a better response. The system is unuseable if we are going to continue to take this long to run the backtests.

I've read the references about what/when NT loads data. That is why I am concluding that there are other problems here. Once I have local data, I should not be spending a lot of time reloading it.

[superarrow]

Hi all,
Jason I have the same and probably an extension to the problem identified by OP. At EOD I playback some charts for testing/sim but when I load up the charts, no data scrolls. The Replay will be running. The charts will charts will show previous date - meaning they are not sync'ed with the playback. So I repair the database (thanks Jessica) and then it works for sometime.

The problem presents itself once I change instruments. I have all the playback data so I would have presumed no issues about lack of sufficient data.

This issue has got progrssively worse over the last week or two. I have two computers with ample resources. This was never a problem before. Only my humble opinion, but I do strongly feel NT caching is an issue. I use another platform (less superior but resource hog) and runs smoothly w/o drama. Note the two apps are never run simultaneously on the same machine.

Cheers,
SA

[NinjaTrader_Jason]

Hello RandyT,

NinjaTrader works as described in the link I provided regarding reloading data. Even if you have data loaded in your database, but then reload the same chart with a different time frame or instrument type, all data will be reloaded. It will also reload if the exact same time frame is used; it will only load from the database as described in the link.

I will forward your suggestion to never reload data which is already loaded in the database to our development team and ask them if they can add this to the list of future considerations for the software.