NT8B10 playback-MR- performance destroyed? - DO NOT USE?!!!

[sledge]

Umm, 7 minute jumps on NT8Beta10 with same setup on Beta9?! MAX CPU usage? 1000x might perform faster! [I'm not sure if I have ghost workspaces from previous 4 Beta10 crashes earlier in the night that I've worked around?!]


This is the only workspace open.

I will not let the laptop run at this pace for days... NT7 is faster than this!





Oh yeah - AFTER hitting pause - I could not RESTART play.. (see other posts http://ninjatrader.com/support/forum...d.php?p=453674 )



I'll try a clean install on everything in my NT8\My Documents tomorrow.... as it's 1am EST... I'm going for some ZZZ's...

[sledge]

At least run on your development environment and confirm the same issues.

I won't install on other machines I depend on until this machine is cleared up and working.

[ my 'warning' is in regards to my new issues with playback market replay ].

Edit: it was running fast earlier as expected on 1 contract. .but in multiple things fell apart completely .

[koganam]

At least run on your development environment and confirm the same issues.

I won't install on other machines I depend on until this machine is cleared up and working.

[ my 'warning' is in regards to my new issues with playback market replay ].

Edit: it was running fast earlier as expected on 1 contract. .but in multiple things fell apart completely .

Are you doing anything at all in State.Terminated in your strategy?

[sledge]

Are you doing anything at all in State.Terminated in your strategy?

No? Why should I? Doesn't get much simpler than market order/set stop loss/target and let it roll..

It worked without in NT8B9 and before?


----
I can't even get out of current contract .. NT7 500x is faster than this current implementation in Beta10. At least it doesn't eat CPU cycles....

[koganam]

No? Why should I? Doesn't get much simpler than market order/set stop loss/target and let it roll..

It worked without in NT8B9 and before?


----
I can't even get out of current contract .. NT7 500x is faster than this current implementation in Beta10. At least it doesn't eat CPU cycles....

You do not need to do so. I was asking because we have had some strange results with State.Terminated because of how it is being implemented so far.

ref: http://ninjatrader.com/support/forum...ad.php?t=83608

[sledge]

I removed everything in My Documents folder and retested with new layout. CPU usage was very high as in the first image.

In NT8B10- at 10:00pm for all practical purposes.

2016-03-23 21:59:21:772|1|8|Execution='5e2c8f21d4de4a1cbe9f6a a45060b295' Instrument='ES 03-14' Account='Playback101' Exchange=Default Price=xxxx.25 Quantity=1 Market position=Long Operation=Operation_Add Order='cdc0a5bce4674551866a9a9fb8689d67' Time='1/21/2014 x:xx AM'

After 2 hours -

2016-03-23 23:58:45:305|1|8|Execution='0394a783805943ecb4ec5b 12540020d2' Instrument='ES 06-14' Account='Playback101' Exchange=Default Price=xxxx Quantity=2 Market position=Long Operation=Operation_Add Order='854bc2f998834f84af7258b05a2d1fb7' Time='4/4/2014 xx:xx AM'

No additional items were running.


In NT8B9 -
From my log files from 3/18/2016 for the same strategy and same layout and charts and same replay dates:

Market Replay START time 8:00 pm for all practical purposes.

2016-03-17 19:59:03:089|1|8|Execution='ea6ea29ddf284841a5dba6 c5a2c3c5b4' Instrument='ES 03-14' Account='Playback101' Exchange=Default Price=xxxx.25 Quantity=1 Market position=Long Operation=Operation_Add Order='7d961db5a0b24067a8e4889cb464b0fb' Time='1/21/2014 x:xx AM'

After 2 hours:

16-03-17 21:57:59:944|1|8|Execution='646486c0d77f4dd8945be9 82daeb9a9e' Instrument='ES 12-14' Account='Playback101' Exchange=Default Price=xxxx.x Quantity=1 Market position=Long Operation=Operation_Add Order='6ed5d85a5ee347e5876b274ae3085b57' Time='12/1/2014 x:xx AM'
20

And this was with streaming 2 HD security cams, and surfing the web.

8 months total loss of performance in Beta 10 in 2 hour test. Both were at MAX Replay.

[sledge]

Here is a video.

I removed all my strategies and indicators.

NT8 Beta10 has issues with the number of items open. It is very limited.

I removed A LOT of tabs and performance was restored to what I expected.

video: http://screencast.com/t/EGJFtenw

note - at the end I could no reproduce the Playback-MR gets stuck issue - but I assume this was because no longer many things open. I had it earlier when I had my full set of tabs available.

[NinjaTrader_PatrickG]

Sledge,

Thanks for the details and the video.

I will be testing this on my end and I will let you know my findings.

EDIT: I was able to reproduce the CPU spike and caused NT8 to freeze when attempting to pause the max playback. I will submit this to the development team for further research. Thanks for bringing this to our attention.

[NinjaTrader_PatrickG]

After further testing, it appears the CPU spike is expected, however, the freezing is not.

The freezing has been resolved in our internal build of NinjaTrader 8 Beta and will be included in a future version.

[sledge]

After further testing, it appears the CPU spike is expected,.

Please explain how it is expected?

What is ninjatrader doing now that it is essentially crippling itself?

This was never a problem on any previous build.

[NinjaTrader_PatrickG]

Please explain how it is expected?

What is ninjatrader doing now that it is essentially crippling itself?

This was never a problem on any previous build.

I should clarify - I meant with the currently available version. In the internal build I do not experience a max load or a freeze, however logically it will still be possible to max the CPU if you ask NinjaTrader to perform an extremely resource intensive process.

The same steps that caused a 100% load and freeze in 8.0.0.10 do not cause either in our internal build.

[sledge]

Thanks for clarifying. Sounds much better. I'm looking forward to the next iteration.

[sledge]

I should clarify - I meant with the currently available version. In the internal build I do not experience a max load or a freeze, however logically it will still be possible to max the CPU if you ask NinjaTrader to perform an extremely resource intensive process.

The same steps that caused a 100% load and freeze in 8.0.0.10 do not cause either in our internal build.

Nothing was fixed in this area. Still the same issue.

Still - massive 70% load in Beta 11 and barely 6 minutes for a jump. If it was jumping 5 hours at 70% load I'd be happy. NT7 is faster than this. Beta9 was the last Beta that Playback Market Replay worked FAST.

I haven't even applied a strategy or indicator.

Disappointed. WAiting for Beta12 I guess.

http://screencast.com/t/QqwmxIbLGg

Using really big bars in the chart, near 100% load.

http://screencast.com/t/fn9ov6n6EE1

--went back to small bars. svchost.exe finally stopped scanning the install I guess - back to 60-40% cpu usage.. but still only 4-5 minutes a "jump" on MAX. 1000x looks like 5-6 minutes. WAIT - wow.. back to 90% usage on the weekend .. 9 minute jumps...

----------------------

BETA9 comparison (as all previous Betas performed)

It's SCREAMING FAST... 5 days + weekend covered in under 2 minutes.

http://screencast.com/t/QdFVoSZ4y

I would venture to say NT7 is FASTER still at this point, but with memory issues..

[NinjaTrader_PatrickG]

Could you provide me with your Market Replay data so I can test using the same data that you are using?

This will be contained within the Documents > NinjaTrader 8 > db > Replay folder.

You can send this folder using a file sharing service such as https://www.wetransfer.com/

Please make sure to include a link to this forum post and also reference ticket 1487733 ATTN Patrick G.

[sledge]

Could you provide me with your Market Replay data so I can test using the same data that you are using?

This will be contained within the Documents > NinjaTrader 8 > db > Replay folder.

You can send this folder using a file sharing service such as https://www.wetransfer.com/

Please make sure to include a link to this forum post and also reference ticket 1487733 ATTN Patrick G.

Currently uploading data as of now.

I did some more testing tonight...

I compared RESOURCE MONITOR with RESOURCE MONITOR from my NT8B9 speedy laptop...

I noticed several things -
1) Reads on daily NT ES files were 100,000 - 300,000 B/secs..vs 2,000,000 B/secs on B9.
2) ms security essentials scanning a ton of crap (WAS NEVER A PROBLEM ON B9 and earlier)... like mrt.exe...
3) svchosts.exe was running 13% for stuff...

I had last weeks MS patches still lurking in the background... I had not updated this machine.

So I did an update...

Well there was some success -- but eventually - things would fall apart and NT8B11 would consume 90% cpu.... I closed all browsers (never a factor before).... was getting 20 minute jumps - but would eventually fall back to 8 minutes and 90% cpu. Now - this is without any strategy or indicators ....

I removed 48 months of data and managed a 5 GB file (down from 20GB) to upload for you and Christopher on my other post [ http://ninjatrader.com/support/forum...ad.php?t=85563 ] (and still had data issues). I did a backup and I am uploading that file to filemail.

What I did see where spurts - the machine did start performing well... but eventually cratered...and started back to 90% and slower drive reads...

I never had to turn off MS security essentials on this machine...but I blocked it from program files/NT8 and in My Documents/NT8 tonight.. still no luck.

This was my fastest machine... until b10/b11...

https://www.msi.com/Workstation/WT60-2OK-3K-IPS-Edition.html#hero-overview

It used to beat out my other machine with Raid 0:

https://www.msi.com/Workstation/WS60-2OJ-4K-Edition.html#hero-overview

both of which used to also crush my ws60 win10 ddr4 with nvme in raid 0.

https://us.msi.com/Workstation/WS60-Workstation-6th-Gen.html#hero-overview