Strategy Analyzer MultiCore CPU

[florida99]

I've read many posts asking NT support questions about multi core cpus and being used in optimization. Just wanted to provide some non-scientific information and there is no intent to comment on anything other than provide some interesting info, and there is no question.

I want to see if two different pcs running the same strategy & strategy optimization but using different data providers would arrive at the same results. I also thought, since they are two different pcs, this would be a good opportunity to start them together and see how they behaved and if they calculate/use the same optimization time needed.

The two pcs are both AMD CPUs, both running Win 7 64 with the same NT7 version. The difference is one is a quad core 3.5ghz with 8gb ddr2 ram and the other is a dual core 2.8ghz with 4gb ddr2.

What I've learned so far is don't draw any conclusions early on.

I can tell you that early on, from when the optimization time remaining looked like it began to settle down, the quad took about 20 minutes, the dual core took about 25 minutes to settle in to a stable minute count but still bouncy seconds, the difference of time remaining was 4 hours. The dual core lagging the quad. Thought I had the smoking gun. At one hour in I took these two screen shots. Don't draw any conclusions yet. At roughly 40 minutes in, the dual core was catching up in a hurry to the quad core and by 1 hour, the difference in time remaining is only about 22 minutes. As I began to write this post, about 1hr 15 min, I noticed the dual core had less time remaining than the quad. As I'm getting ready to post this, roughly 1hr 30 min, the quad has 9 hrs remaining and the dual core 14 hours. Obviously there is much still being calculated. I'll post again as this progresses.

[florida99]

Thought this was a good pair of screenshots to show why still can't draw any conclusions comparing multi core cpus yet. At 2:09 in, the quad core is back to 22+ hours remaining and the dual core is showing 9+ hours remaining.

[NinjaTrader_AdamP]

florida99,

Thanks for the experiment.

I would recommend timing these outside of NinjaTrader as well. The times quoted change frequently, and the only way to know exactly how long something took is to time it yourself.

Please let me know if I may assist further.

[florida99]

Thank you for the reply and suggestion Adam. Hadn't considered the NT elapsed time wasn't actual time elapsed but I now see it in the screen shots.

Also wanted to note, I caught the quad core 'time remaining' actually counted down to 0 hrs 0 mins 0 seconds and the clock started over again at 23:59:59 and began adjusting itself again. My guess, a part of the process finished and the next part began.

[florida99]

Wanted to update this post with results so far. The quad core finished the first optimization right at 15 hours after beginning.

While the quad core is used exclusively for trading, the dual core is a workhorse machine, used for email and other home office functions. It was unavoidable but the dual core had to be heavily used during the optimization. The dual core elapsed time and time remaining function stopped counting at roughly 7 hours in and NT was left running. The machine was being heavily used at that time for several large file downloads and other office functions. When the quad finished and posted its results the dual core optimization was stopped. The dual core results were posted in the strategy analyzer but the elapsed time/time remaining was showing times. The results were very similiar but not an exact match, which I expected. Clearly something else was going on and the test was discarded.

To be fair, the quad was left alone for the entire 15 hours but the dual was not.

A different, less strenuous test, that wouldn't take 15 hours, was started again on both machines. The quad finished in 4 hrs 46 minues. Per Adam's suggestion, attention was paid to the real clock time. It was started at 3 am and the actual ending time was 7:46 am. The dual core is still working as of this writing. Again some files needed to be downloaded as the optimization was being run. When finished, a third test will be run on the dual core so it will be able to work 100% on the optimization problem.

My only conclusions thus far: a dedicated machine to optimize is better than one being simultaneously used for other everyday tasks. Think we all can agree more memory is better than less and a faster processor is better than a slower. I will reserve opinion on the number of cores until we see further results.

I also have an AMD 1055T six core machine, cpu running at 2.8 ghz with 4gb ddr2. The quad has a 3.5 ghz cpu with 8gb ram. Think this is a better multicore test match. The same optimization is now running on the six core.

[florida99]

For those wanting to know if the number of cores make a difference during optimization, looks like the answer is a definite yes. Greater number of cores beat out faster cpu speed + more memory. Below are two screenshots from the 6 core cpu, a 2.8 ghz AMD cpu with 4 gb ddr2 ram, same NT7 version and same optimization strategy. The AMD quad core it was compared against is a 3.5ghz cpu with 8gb ram. (All three motherboards use identical chipsets, AMD 785G, all three use pc2 6400 800mhz ram and all three are FSB 2600MHz Hyper Transport (5200 MT/s))

The first screenshot, I was just trying to capture the cpu utilization. I saw about 63% peak, the screenshot caught about 52%. I posted the second screenshot because I thought it was just an interesting catch, the processor is just throttling down from 50% or so utilization down to 5%. As far as the times, the quad took 4 hrs 46 minutes to complete, the six core took 4 hrs 41 minutes, 5 minutes better. The results using the same data provider were close but slightly different. I think that's the subject of another post.

The dual core cpu was stopped at the end of 6 hours on the second pass as the pc had been used for other tasks during optimization. Given a clean reboot and restarted on a third pass 1 hr 5 minutes after the six core and is still running (now 4 hrs 7 mins), with no other usage. By one hour to go, both the quad and six core pcs were stable as far as time remaining seconds counting down, though as Adam pointed out, it wasn't representative of real time. The time remaining was counting down faster than actual time. You can see it in the two screenshots below, compared to the system clock. The elasped time was accurate on all the tests.

As far as the dual core, it does not appear to be within an hour of completing. I will post one last time when it does.

In conclusion, imo, it looks like number of cores does make a difference as far as shortening calculation time. And if you ever want to get finished, don't use a dual core machine. LOL

[florida99]

Shot of the dual core still working. Actual start time was 9:09 am. The real elapsed time is 5 hrs 19 mins, matching the optimizer elapsed time. The cpu utilization was caught at 95% but is spending most of its time pegged at 100%. Not looking like it is close to finishing anytime soon. No other work conducted on the machine since this start.

[florida99]

The dual core finally finished the task at just under 19 hours after starting. Having those extra cores did make a big difference, about 14 hours.

ps, the original update that was posted here was removed since it added nothing and the final was placed instead to shorten the thread.