High Volatility Environment and NT Performance

[Zapzap]

PC clock sync is of utmost importance.

Speaking of which when I run NT, my PC clock can get out of sync by 10-30 secs in like 10-20 minutes of running time, which can confuse not timestamped tick charts.

I use atomic time sync with NIST servers, so I advise everyone to check it too.

I don't know why it lags so badly though, I only know it does.
(And at 30secs+ Ninja-Zen connection reconnects.)

[NinjaTrader_Bertrand]

You could setup a tool to autoSync you each x mins or so, effectively taking care.

[NinjaTrader_JoshP]

Zapzap,

From personal experience, the motherboard keeps track of the clock via CMOS and unfortunately some motherboards just don't do a very good job at it or the motherboard battery may be weak. I have had a computer that lost sync in excess of couple hours for every day I had the computer on.

Just thought I throw that out there.

[aviat72]

aviat72,

There is no point theorizing on what could/could not be the case. Please check your CPU usage in the Task Manager when you see such a situation. Only then will we be able to know what is happening. If the CPU is maxed out then to proceed will be to consider some of the performance tips Austin has provided earlier. If the CPU is not maxed out then performance is not the issue to blame and instead it could be either that was simply what your data provider was offering or network congestion or dropped packets, etc.

If you are worried about horsepower please be sure your computer meets minimum system requirements: http://www.ninjatrader-support.com/H...lationAndSetUp

Josh

I appreciate the quickness with which you response; but I am hoping that you put some thought into the issue before posting. The standard method of isolating such problems is to postulate some hypothesis based on observed symptoms of the problem and then finding evidence to support or disprove the hypothesis.

Regarding CPU usage, there are multiple threads open here about how NT uses up CPU (25% on a quad core) even when there is one chart open. So how is that a reliable indicator?

Have even tried talking to development and showing them the chart where the instrument which gets trade ticks is lagging but the instrument with a fixed number of ticks (one per second) is not.

The architects/developers familiar with the underlying software architecture should be easily able to point out whether the delay is in the data-feed or it is CPU processing power which is the culprit.

Once we have a clear idea of where the problem is, I can try and design a systematic workaround.


A lot of us here spend quite a bit of time and energy in identifying problems, doing a first level analysis, and then collecting that information and posting it. In return I expect support to show some respect to the effort being put in, and perhaps do a bit extra. At the end of the day all it will do is make NT more usable.

[NinjaTrader_JoshP]

aviat72,

There is no need to bring in development at this point in time and thought is always put into my responses. If one chart open is taking you up 25% that is definitely your CPU being pegged on a quad core. Charting uses 1 core and being at 25% means you are pegged at max for 1 core. In all of your other posts you have provided no evidence that your CPU was actually pegged which is why I asked for your computer specs earlier. Only now have you provided evidence of your CPU being maxed and as such moving forward I suggest you consider the performance tips Austin has provided earlier. I suggest you first remove all indicators from your chart and determine that it is not any of the custom indicators causing issues. If you have no issue with no indicators then comes the task of slowly adding the indicators back to isolate which one is causing you issues. When the source of your maxed CPU is determined, if necessary, then development can be brought in at that time to address any outstanding issues.

[aviat72]

Josh

I never said that this particular CPU was maxed out. I just pointed out that there are so many underlying issues with regards to NT & CPU performance, that CPU performance is NOT a reliable metric to draw conclusions from.

Pointing to CPU horsepower as the issue would have been fine if the entire set-up was freezing; however that is not the case here. The chart with the internals were keeping up. And NT was responsive to mouse-clicks. I have seen NT die due to CPU issues and this was not the case here.

Hence the need for more detailed analysis and the suggestions to involve higher powers instead of the usual script.

[NinjaTrader_JoshP]

aviat72,

We are definitely not on the same page then. Please clarify what exactly the facts are.

Was the CPU pegged or was it not?

If it is not pegged it is definitely not a performance issue and all talk about CPU and performance is irrelevant for this case. There would be no queue of ticks without your CPU being pegged. Only when your CPU is incapable of keeping up with the events to process would a queue start forming and keep your CPU pegged and thus cause delays.

[aviat72]

Josh:

I did not check whether the CPU was pegged or not (the market was crashing!).

I do remember that all the charts were updating, the mouse and other drop-down menus were responsive, there were no flashing screens etc. I was able to take the picture without a problem at all using the menus. This is not the typical behavior when CPU is pegged. My second PC has a non-stop CPU peg on 1 minute charts (it is fed via IB) and it takes seconds and sometimes even a minute for NT to respond to a mouse click.

However the panel showing instruments where a huge amount of ticks were coming in were falling behind. Instrument with one update a second were fine.

This could simply be a data-feed issue. I would find it surprising though that the tick feed falls a full two minute behind the internals coming from the same data-vendor. But weirder things have happened; the problem could be with the exchange itself thought I doubt it since IB feed was current (i.e. the exchange was sending out the data on time).

Or it could be an issue related to NT's architecture. There can be a variety of possibilities. For example, there could be a time-slicing thread scheduling algorithm implemented internally which would move on to the next instrument after processing a large number (say 100) of ticks one instrument; this is often done to ensure that one task does not hog the CPU. This would not show up as an unresponsive NT but it could result in one instrument falling behind since for one tick of the internal you might be getting 500 ticks for the future contract.

However unless we have a definitive answer I can not workaround it. There are multiple options I articulated in the first post. If your team is definite it is a data-feed issue I can ask IQFeed about what happened. If it is a software architecture issue, perhaps NT internally could do a review. If it is a pure horsepower issue, I can fall back on a filtered feed. I do not expect any system to keep up with 500 ticks per second with 10 tick based indicators working live.

But I need clarity.

[NinjaTrader_Brett]

Hello,

This is Brett following up, I dont believe we have taken a look at your trace files yet also I would be interested to taking a look at your workspace to see what your running.

Can you please send your trace and log files so that we may take a look at the trace files.

Can you also send your workspace file so that I may attempt to duplicate your issue on my side.

You can find these files at the following locations, please attach these to a return email and in the subject line but ATTN Brett and reference this forum post.

Start Menu--> My Documents--> NinjaTrader 7--> Trace--> trace.20100517 for today. (trace.yyyymmdd for any other date)

Start Menu--> My Documents--> NinjaTrader 7--> Log--> log.20100517 for today. (log.yyyymmdd for any other date)

Start Menu--> My Documents --> NinjaTrader 7-->Workspace-->Your workspace name.xml

[NinjaTrader_JoshP]

aviat72,

Spoke to development, but unfortunately we cannot speculate anything. What they suggest is if you see this happen again to bring up a T&S window and see the timestamps of incoming tick from there versus your chart. If you have updates on T&S but not on the chart then it can be said you are running into performance issues on the chart. If T&S and chart represent the same timestamps then it is more likely data feed issue. If you close the chart and just had T&S and timestamps remain the same, it is most likely that was simply what the data feed was pushing down the pipe.

[LonelyTrader]

aviat, NT7 is not a redesign -- it is simply a revision of 6.5. The guts are pretty much the same. It does not take advantage of multithreading, so your charts will be pegged to one core. You're better off running this software on a very fast single core CPU and even then you'll probably be disappointed in high volatility environments. Note I didn't say "very high", because this problem occurs frequently enough that several people I know are still unwilling to trade real accounts on this latest iteration. The architecture needs a complete overhaul for the software to do what is touted in the marketing and this probably means a large increase in the price of the software. There are many people who have spoken to issues similar to yours and I haven't the foggiest idea why the support folks haven't acknowledged this. My guess is that you will not find a workaround with a multicore CPU while trading. Backetesting should be a different story, however. I believe you can partition each scenario and run parellel on different threads.

Until this stuff is fixed, the only people who will be interested in this software are coders and indicator/system hawkers. Or people who just have one chart and one DOM on their desktop.

[MXASJ]

LonelyTrader, this is where you should point out all the multi-threaded alternatives that handle trading and charting. Look at X_Trader... oops single thread. Or CQG... oops single thread again. Well surely RTS Tango is multi-threaded as its built on Unix... oops. Single threaded. And it doesn't chart.

Different tools for different traders. Number of charts open in Ninja right now while I manage three open positions? 0.