Optimization time

[bobby1001]

I sent it. Will you reply here, or to my e-mail?

[NinjaTrader_Dierk]

To your email

[bobby1001]

All right, thanks. I also sent a blank e-mail should the first not get through for any reason. I have a new setup and things are still a little touchy.

Thanks.

[Jenny]

Hi, NinjaTrader_Dierk,

I really want to know the result, what is the solution for this question? 4 variable take 3 minutes but 5 variable 6~22 hours. Only 1 variable different, why running times are so much different?

And an intel quad processer computer sounds very powerful. 4 variablesX10 possibilities=10X10X10X10 only need 3 minutes.

Thanks && Happy new year
Jenny

I am optimizing 7 variables. If I optimize over 4, I have a drag time of about 6 to 22 hours. All variables have between 6 to 10 possibilities. If I only do 4, it takes about 3 minutes. If I do more, the please wait box time to go fluctuates wildly, and the process does not seem to work.

I'm running an intel quad processer with 4 gbs of ram, so I don't think it's a system issue.

Please advise as this wait period does not make sense to me, and it is to long to wait.

Thank you.

Bobby

[KBJ]

You might check to see if the number of Page Faults greatly increases when you increase the number of variables being optimized.

This may be checked by using the Windows Task Manager, clicking on the "Processes" tab, and then "View", "Select Columns", and selecting "Page Faults".

Then check for number of page faults before and after running each series of optimizations.

If the number of page faults is not proportional (within reason) to the total number of combinatations (calculated from the product of the numbers of possibilities for each variable being optimized), this would indicate that the optimizer is running out of working set memory and creating I/O's to the page file as part of its virtual memory utilization.

Each page fault causes an extra I/O, so if this is occurring, this would be greatly slowing down the optimize run.

Check out this definition of virtual memory, which describes what page faults are all about: http://en.wikipedia.org/wiki/Virtual_memory

[Jenny]

Dear KBJ,

Thanks for your information. I will monitor the page faults.

I am running a program, about 3.5 months data by minute, only the back test take about 3 minutes. That means 120 combinations need about 6 hours. Do you think that is normal?

Thanks a lot
Jenny

[Pete S]

Jenny:

That doesn't sound normal to me. My primary strategy, which I run on 5 minute bars, takes .05 seconds to run one iteration as timed in the code. You are saying it takes 3 minutes for one single iteration? If so, you must be using a bad algorithm in your code somewhere. I'd suggest doing a little manual profiling of your strategy code.

[NinjaTrader_JoshP]

Sometimes I find that running backtest/optimizer will draw upon your connection to try to get data. I find it runs much faster if you first download all your data and then disconnect from your datafeed and drive your backtest from your database only.

[Jenny]

Thanks for Pete S and Josh replying my question.

I had downloaded data, but it is still the same.

I had some other code, it works fast. Maybe I have too many Variables? Here is the session:

#region Variables
private int siganlSMA =1041;
private int pe =8;
private int p1 = 44; // Default setting for P1
private int p2 = 205; // Default setting for P2
private int b1 = 81; // Default setting for B1
private int b2 = 3; // Default setting for B2
double Low2=0;
double High2=0;
double Low1=0;
double High1=0;
int LowBar2=0;
int HighBar2=0;
int LowBar3=0;
int HighBar3=0;
#endregion

In my code, I also use :
MRO(delegate {return (CrossAbove(SMA(5), SMA(10), 1));}, 1, 260);

Would you please give me some advise?

Happy new year

[NinjaTrader_JoshP]

Sorry Jenny. I am out of ideas as to why your optimizing is taking so much time.

Here is a benchmark from my system you can check against:
SampleMACrossOver backtested on 1min from 1/1/2007 to 1/4/2008 took 6 seconds.
My computer is 1.84ghz with 1gb ram on WinXP SP2.

[Pete S]

Extra variables will not, in general, make your code run slower. What really makes your code run slower is two main things:

1. Excessive memory allocations
2. Bad algorithms -- algorithms where the performance decreases in a worse than linear fashion as the input size grows

You definitely don't want to be doing this in your code: MRO(delegate {return (CrossAbove(SMA(5), SMA(10), 1));}, 1, 260);

What you should do instead is, in the variables region, do this

private SMA mFastSMA;
private SMA mSlowSMA;

and in your Initialize method:

mFastSMA = SMA(5);
mSlowSMA = SMA(10);

Then you would reference them like

CrossAbove(mFastSMA, mSlowSMA, 1)

in your code. This will not matter for running live, but will make a noticable difference when running thousands of iterations on an optimization.

Note, however, this alone will definitely not account for what you are seeing. Like I said in my last post, a little profiling is in order so you can find out exactly what is happening. The first file I have attached is an instrumented version of the standard optimizer. To install it, save the file to your Ninja Trader 6.5\bin\Custom\Type directory and recompile your strategy. Then, when you run your optimization, pick 'Default with Timing' as the Optimizer. Post the output here and I will help interpret it; it will look like this:

FCSX: SampleMACrossOver: starting optimization at 1/6/2008 10:34:34 AM: total iterations: 820
FCSX: Iterations: 100/820 in 0.1 minutes (0.03 sec per iteration)
FCSX: Iterations: 200/820 in 0.1 minutes (0.03 sec per iteration)
FCSX: Iterations: 300/820 in 0.1 minutes (0.03 sec per iteration)
FCSX: Iterations: 400/820 in 0.2 minutes (0.02 sec per iteration)
FCSX: Iterations: 500/820 in 0.2 minutes (0.02 sec per iteration)
FCSX: Iterations: 600/820 in 0.2 minutes (0.02 sec per iteration)
FCSX: Iterations: 700/820 in 0.3 minutes (0.02 sec per iteration)
FCSX: Iterations: 800/820 in 0.3 minutes (0.02 sec per iteration)
FCSX: finished optimization at 1/6/2008 10:34:53 AM, iterations: 820, total time: 0.3 minutes

The second file I have attached is an instrumented version of the SampleMACrossover strategy. It's an example of how you might instrument your own code to find the areas that are taking too much time. You'll also note I put in a section to demonstrate what I explained above; you can see it is roughly 100% faster in a backtest.

I've spent a bunch of time doing this for myself and my primary strategy runs 10 times faster than it did before optimizations.

[Jenny]

Pete S,

I really applicate you help. I already modify my code according to your advise, Here is the result:

$NZDJPY: LowHigh12e: starting optimization at 1/7/2008 12:08:02 AM: total iterations: 4
$NZDJPY: finished optimization at 1/7/2008 12:10:30 AM, iterations: 4, total time: 2.5 minutes

Thanks a lot
Jenny

[Pete S]

So each iteration now is taking ~37 seconds, that is still a shockingly long time compared to anything I have seen. I'll note that I don't know anything about trading Forex & if that has anything to do with it.

I'd recommend one of these things:
1. Follow my example from the MACrossWithTiming strategy. Take a look at the code, put those timing blocks around each individual section of your code. Find out where it is spending its time. Then you can focus on that section of code or post that block here for help.

2. If you don't mind sharing your strategy, you can email it to me and I'll take a look at it.

I suspect there is one section of your code doing something unintended.

[Jenny]

Pete S, I have result after embed your timing code, but it is too big to post here. Can I sent to your private email address?

Thanks for time

[Jenny]

Pete, I already send the result to your email.
Thank you very much