Optimizer and threads

[neb1998]

Optimizer doesnt always use all threads available...why is this?

[NinjaTrader_Bertrand]

Are you using a bool or enum in your code with the GA? Then it would be limited to one thread only unfortunately - to remove this limitation you can use the regular optimizer or instead switch to int inputs for example.

[neb1998]

Bertrand, i knew of that limitation and no i am not using either. I dont know what the issue is.

[NinjaTrader_Bertrand]

Please clarify what you attempt doing exactly - what kind of system are you using? How many cores are available? Are you working with the GA or default exhaustive optimizer here?

[neb1998]

Simple system. I have one long entrance, and 2 exits, a stop and a limit. Managed Exits.

Genetic Optimizer, i have 4 cores available. (physical)

[NinjaTrader_Bertrand]

It should use all available cores then, if there's no bool / enum input in your strategy used - do you see the same when optimizing the SampleMaCrossOver from us?

How do you exactly determine it would use less cores?

[neb1998]

I just ran the Sample and it only is using ~~15% of CPU on a 8 core machine, which is 1 core.

Taskmanager

[NinjaTrader_Bertrand]

On a 8 core CPU you would see 8 iterations / threads working at the same time to complete the full optimization result needed - that does not have to mean that your CPU / core load shown in the task manager is at 100% all the time.

[neb1998]

Right but one core in the performance monitor goes to 100% usage, which is 15% of the total. So only one core is being accessed. During a long optimization the # usage should rise above 15%, to 30% for 2 cores, etc.

[NinjaTrader_Bertrand]

We could unfortunately not reproduce your observation here - the GA used all 4 real (non hyperthreading) cores available as we ran SampleMACrossOver in our testing. Do you have another strategy script installed that would perhaps trigger the bool / enum limitation we already spoke about?

[neb1998]

So if another strategy exists on my machine that uses bool or enum then it disables multithreading for all other scripts?

I cant verify if i do now but its possible. I never use enum. But i might have a bool parameter somewhere, although i have been replacing them with variables anywhere i can.

[NinjaTrader_Bertrand]

I think we can rule that idea out, I tested and the bool in another script did not influence the currently tested one - can you please send me your config.xml file to support at ninjatrader dot com and my Attn? You can find it under MyDocuments > NinjaTrader 7.

Thanks,

[JustCallMeDan]

I too have stumbled on this behaviour. I recently upgraded to an AMD Phenom II X6 1100T on my desktop (among other upgrades) specifically to significantly improve optimisation times over my old Intel Core 2 Duo laptop, by introducing new cores at a greater speed.

I find only one thread running at a time in while performing genetic optimisations (I am using a gadget on Windows 7 that informs me constantly of my core usage. It seems NT is just using random core for one thread at a time.) no bools or enums either. Exhaustive optimisations multithread perfectly, with one thread per core and all cores in near 100% use.

Now, my strategy hooks into DLLs (64 bit and threadsafe) etc, which I know is completely outside what you can support, although I'm confident this is not the root of the issue (especially as it multithreads fine with an exhaustive optimisation). My modifications confirm that I only ever have one instance of the strategy running at any one time with genetic optimisations, as I have it specifically reporting this in the log.

In terms of what you CAN support, this behaviour is reflected while optimising SampleMACrossover.

Any clues? I hope the genetic optimiser isn't using proprietary 'hyperthreading' (i.e. only working for some Intel CPUs)? Stab in the dark there...

[NinjaTrader_Bertrand]

JustCallMeDan, thanks for the report and post - I'm still chasing this issue you also experienced - could you please contact me via Help > Mail to Support and also include your config.xml file from the MyDocuments > NinjaTrader 7 folder in addition to the trace / log files?

Are you on latest NT7 R2 as you see this outcome?

[JustCallMeDan]

Hi Bertrand,


Will do. Yes, this is using NT7 R2 (64-bit 7.0.1000.2)