System.OutOfMemoryException

[Edward Kunafi]

Hello Support and any Windows XP expert out there,

I am running Strategy backtesting with 3 months of TF data, no charts are open. My system is XP 32 bit, with 3GB RAM. This is fully available space, graphics memory and IO space is outside this. At the Control Panel->System->Advanced Performance Options->Advanced Tab I have verified processor and memory are optimised for running programs, also I have set virtual memory paging file size to be 8GB, much larger than RAM for all drives, this ensure paging file will never runs out of space. In fact I have set maximum size option to 16GB.

As the backtesting progresses, Ninjatrader.exe process increasing use more memory starting from a few 100MB to more than 1GB. When it reach about 1.2GB, NT reports "System.OutOfMemoryException. Pagefile reach about 1.7GB, and the system still has 1.3GB available ram. Please check out three relevant screen captures. I do not run any other apps while doing backtesting.

For XP 32 bit each applications are able to have up to 2GB space, why does NT runs out of memory in this case ?
What other XP tuning one can do to make sure it will have 2GB available to it?

Thanks.

Regards.

EdwardK.


Thanks

[NinjaTrader_Bertrand]

Edward, the info is unfortunately not correct - the max RAM limit available for 32 bit process is around the 1.2 GB you experience in your testing.

With NinjaTrader 7 the memory footprint would be much smaller, thus it would not crash at this task as it would not run into this 32 bit native Windows limit.

[Edward Kunafi]

Hello Bertrand,

Can you let me know where you get the info on 1.2Gb address space limit for 32 process?

My reference is from Microsoft itself clearly say 2GB as described in the link below.

http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx

Thanks.

Regards.

[NinjaTrader_Bertrand]

Edward, this is the theoretical limit yes - in reality around 500 - 800 MB as used and allocated by the OS itself so it would leave about 1.2 GB available for the NinjaTrader 32 bit process, not taking into account other apps working.

[Edward Kunafi]

Hello Bertrand,

The OS does Not use part of user apps virtual address space of 2GB. Even if there is no more RAM available, it will have to ensure 2GB space by swapping parts of the virtual space to disc, hence the use of PageFile. If this happens only slowing down will be experienced by the user. In my case, there is still 1.3 GB available RAM.

I would presume stack space used by NT will have already been included in the reported memory usage.

I am afraid I do not understand your explanation. I still think there is other parameters in XP that is limiting user ram space.

Thanks.

Regards.
EdwardK.
EdwardK.

[NinjaTrader_Bertrand]

Edward, unfortunately I'm not an expert in Windows XP architectures, however the 1.2 GB is the practical limit experienced - http://blogs.msdn.com/b/tom/archive/...processes.aspx

[Edward Kunafi]

Hello Bertrand,

Thanks for the link. It helps me understand the issue.

How much typical reduction of memory usage can be expected if run on NT7?

Thanks.

Regards.

EdwardK.

[NinjaTrader_Bertrand]

You're welcome Edward, good - hard to generalize, however for example a simple 10 days 24 hrs tick based 4 Range ES chart loaded on ZenFire would reduce resources used by NT7 about factor 2 to 3.

Since NT7 would also support true 64 bit architecture where possible the advantage using this option is then even higher.