Strategy Analyzer Optimization - 32 Bit vs 64 Bit

[TheTradingMantis]

I know there have been a few related posts here, but wanted to add some comments about my experience.

On the positive side. Optimization in 7.0 Beta 20 is much faster than in 6.5. The 32 bit version works well on my primary backtest computer (Intel i7 quadcore with 6 gb memory).

On the negative side. The 64 bit version has some serious memory management issues. Anything but the simplest optimization soaks up all available physical memory and brings my computer to a standstill. Memory usage exceeds 98% and spends most of the time swapping to disk with CPU utilization then dropping to under 10%. It actually takes much, much longer to run the same optimization on the 64 bit version.

[NinjaTrader_Jason]

Hello gregschr,

Can you please perform an optimization using the SampleMACrossOver strategy. Does the same optimization take longer using the 64 bit version as well?

[TheTradingMantis]

Hi Jason,

The SampleMACrossOver strategy optimization ran fine on the 64 bit version. I used 3 years of 1 minute data, 1500 iterations (20 on one parameter 75 on the other parameter).

I did not have a memory problem running it.

Greg

[NinjaTrader_JoshP]

Greg,

Since SampleMACrossOver is working as expected it most likely is something that would need to be isolated in the particular strategy you are using. I suggest stripping away the strategy to its barebones and then slowly adding back layers of complexity to find where exactly it starts having issues in the strategy code.

[NinjaTrader_JoshP]

It should also be worth noting that 64-bit architecture inherently requires more RAM. This is not something that NT can control but is rather simply how 64-bit applications work. It is possible that you run into a strategy setup that would be fine on 32-bit, but would then start requiring memory swapping as you do the same in 64-bit.

[kdoren]

@gregschr: I'm running NT7 64-bit under Win7x64 with 12GB of ram. The only time I've seen the symptoms you describe during optimization (high memory utilization, low CPU utilization) was when I was calling a Matlab DLL which turns out not to be thread-safe. Since NT7 runs optimzations under multiple threads, you can run into problems that you wouldn't have in NT6.5, which look like NT7 problems but they're really not. Problem went away when I put lock statements around the DLL calls (which of course makes some threads wait for DLL access so I can't use all of my CPU power, but that's not NT7's fault). FYI the strategy I'm optimizing uses 4 months of 1-min data for 100 symbols in a single strategy, but when you think about it that really isn't that much RAM in a modern PC; I think tick data is where you might start hitting the limits on RAM.