New box, upgraded Windows, NT7 runs worse than on the old setup. Keeps hosing up.

[AMATX]

So, spent hefty $$$ on a new workstation, upgraded to latest Win10Pro, latest NT7(in process of migrating to NT8), latest IQFeed for live quotes. Basically latest EVERYTHING and the damn charts are hosing up pretty much everyday now. I've even cut back on the number of charts and that hasn't stopped the problem. Today, appears to have hosed ~14:10 Eastern time. Not much in log file, occasionally some disconnects(usually w/reconnects) in the trace file, but that's about it.

CPU not an issue; plenty fast cores. Even jacked up priority on NT7, for what that's worth. Gobs of memory, too, so nuthin' there, either.

This ran a LOT better, even tho very slow to load, on my old Win7Pro workstation, with slower cores. NOT good.

I've attached today's log & trace files, lemme know what's going on here, cuz this ain't gonna cut it. Mid-late afternoon, EVERY day now.

Other than that, it works great...

[NinjaTrader_RyanS]

Hello,

Thanks for posting.




This is expected when running a very large workspaces with NT7. NinjaTrader 7 is a single core application as it was designed over 10 years ago before multi-core processors became popular. This means it can only utilize one of your CPU's cores to process data/calculations. In your case you have 16 cores, so NinjaTrader 7 can only use 1/16th of your overall CPU power. So if NinjaTrader is using 6% of your CPU, it is being maxed out.

To prevent this behavior you'll need to reduce the load your workspace places on your system. The least intrusive way is to reduce the days loaded on tick charts/indicators to less than 5 or 3 if possible. The next would be removing charts and indicators until the issue is remedied.

NinjaTrader 8 is a multi-core application, so it is not limited in such a way.

Also, we have removed your log and trace from the post for security reasons. These files can contain personal information. If you wish to provide log and trace files for investigation, please write into platformsupport[at]ninjatrader[dot]com for assistance.




Please let me know if I may assist further.

[AMATX]

Hi RyanS, thx for getting back to me. I'm simmered down a bit, now...

Yes, W-2145 chip = 8 cores/16 processes. I've been keeping Taskmgr open to monitor the system and while I haven't been closely watching NT7, I recall it using low to mid-single digit cpu. I think the reason NT7 works ok in mornings, but hoses later in the afternoon may be due to loading up it's memory as the trading day progresses. Interestingly enough, if I restart NT7 in the early afternoon, it seems to work fine though the rest of the session. I don't have a lot of examples of doing this, so can't say for sure whether this really works or not, but it has a time or two.

Can you find out if anyone there at NT HQ has knowledge as to whether turning OFF hyperthreading will affect performance? If so, good or bad? 8 real cores is enough for my workload, so maybe that'd boost the threshold up to 12.5% before a real core/cpu is maxed out.

The heaviest cpu/memory apps I use are all single-threaded, which is not good. Turning off hyperthreading is something I've thought about, but not yet tried.

I'm unable to do much about charts right now, as for some of them, I need a VERY significant lookback period, so no relief there. However, since NT8 is a real PIA to migrate to for -some- of my charts/indicators(gonna be a while on those), I'm in the process of moving the charts that only use stock NT indicators over to NT8 to do some load balancing. Not done yet, but in view of your comments, I may try to move some more of them over tonight, so I can see what happens tomorrow.

More when I know something...

[NinjaTrader_RyanS]

Hello,

Turning off hyperthreading could improve the performance slightly, though I would expect this to cause performance issues with your PC in general, and possibly compatibility issues with Windows 10.

Other possible alternatives would be to split your current workspace into two different workspaces and switching between the two, though this may not be a realistic alternative depending on your trading strategy.

Below are links to all our performance tips for your reference:

https://ninjatrader.com/support/helpGuides/nt7/performance_tips2.htm

Please let us know if you have any other questions.

[AMATX]

Ok. Thx for pointer to NT7 performance tips. I hadn't considered changing the update interval. Might be worth doing on some of my lesser charts that don't need more immediate response. Other tidbits I was aware of and doing, so not much more there. I need ALL of the charts available all of the time, so switching between workspaces ain't gonna cut it.

I do have a question or two about multi-threading in NT8, but I think I'll open a new thread in the NT8 section for that.

I'm probably gonna try turning off Hyperthreading in the BIOS, and will report back when I know something. I do not think it will negatively impact performance on my box, considering what software I'm running(including NT7).

[AMATX]

Well, off-shift non-Hyperthread testing looks good. When I turned it off and rebooted, Taskmgr correctly shows 8 -real- cores, which is what I want.

Booting up NT7 was faster(altho it was quite good before), and NT7 is/was able to use significantly more cpu than before, which is what I want.

However, even off-shift, it now hoses if I try something simple, like changing the days lookback for a single chart. I also deleted several charts, but when I tried to save the new config, it just HOSED. Nothing in log/trace files.

Since I did this just now, I'm wondering if NT7 has a problem loading -today's- data, altho the .ntd files appear to be on disk. So, dunno what's going on, but I'll retry after a day change and see if that makes a diff or not.

In the meantime, I'll move some charts over to NT8 to offload some of the work and give it a try on Monday.

As mentioned above, this issue seems to be only occurring later in the trading day, which makes me wonder about screwed up internal NT7 memory management not being able to hold the amount of data picked up during the day(or maybe not being able to process it).

Any rate, seems pretty crappy to me, as this oughta work and did on a much slower 4-core Win7Pro box vs. this Win10Pro 8-core box with much faster clock speeds and a bunch more memory.

Regardless, I'll probably come up with some way to get it to work; I just hope it doesn't take forever...

I'll report back next week, when I know more, but for now, no HT and 8 real cores

[AMATX]

Looks like simplest, easiest solution for now is to move what I can from NT7 to NT8, which frees up more resources for what stays on NT7. There is clearly some sort of memory management/capacity issue with NT7 and my charts, but I don't wanna get off on some weird tangent trying to fix it, so just gonna do the above.

Turning off Hyperthreading appears to be the way to go, so a little more NT7 ---> NT8 chart migration -should- result in minimal or no issues starting this coming Monday.

[AMATX]

Busy day today in markets, while tanking.

NT7 worked better, didn't hang up/hose up, probably due to more cpu being available because I turned off Hyperthreading. So, that appears to be -one- part of fixing NT7 issues. Dunno about rest, yet. Going to continue moving some charts over to NT8, leaving only stuff that needs a rewrite left on NT7.

So, at least everything was up today and ran ok. Actually, better than before. For anyone who has most/all of code that is still single-threaded, I'd recommend experimenting with turning off HT; seems to have worked for me.

Another possibility for my situation might be to run the same version of NT7 on new box as is on the old box. Don't know if I can do that or not, but has occurred to me. Could be something got 'fixed' in NT7 and is detrimental to my setup. But, that's for another day.

[sandman]

I'd like to use your suggestion and turn off this Hyperthreading, but how the heck do I do that? Is there a simple box somewhere with a check mark to turn on/off???

sandman

[AMATX]

Hi - HT is usually controlled thru the BIOS settings for the MB. And, get this, some boxes don't allow you to change the factory HT settings, so your mileage may vary.

That said, I'm running a Lenovo workstation and it was very easy to switch HT off in the BIOS:

* Cold boot of box, tap F1 as it boots, which gets one into the BIOS panels.

* Find a panel dealing with cpu related stuff.

* Turn HT off(or back on, depending...).

* Save BIOS settings and finish booting up Windows.

That's it. Simple and easy.

To verify, I opened Taskmgr, then opened Resourse Mgr and looked at the cpu screen. That screen gives a breakdown for each cpu that Windows sees. This number went from 16 cpus listed(with HT on) to 8 cpus when I turned off HT. I'm using an Intel W-2145 chip, which has 8 real cores. HT turned ON adds 8 threads to that, which to Windows appears to be a total of 16 cpu cores with which to work. Hence, Resource Mgr is an easy place to verify what's going on...

Box running mucho better now, so if you have mainly single-threaded, single-core apps, this is worth trying.

Big hogs on my box are Thinkorswim(single cores only; a real pig. I run multiple copies) and NT. Migrating from NT7 to NT8, but because of various issues, this is gonna take a lot longer than it should, if ever. Also, NT8 is apparently not truly multi-threaded, so if you have a huge NT8 workload, you might wanna buy a box with a LOT of cpus on it, to spread the workload around. Fortunately, NT doesn't seem to require really fast cores(like TOS does), so a bunch of slow cores would probably still work, if you wanna save a few $bucks$ on which cpu chip you use.