Backtesting month by month much faster than 3 months together

[ivanjocic]

Hi,
for b16 I've created a very stupid strategy that was demostrating that more orders had to be handled (1 entry per direction) performance was degrading exponentially. For example if I generate 10000 orders per month it took 5 minutes to backtest, but when I backtest week by week with 2500 orders per week it took 20 seconds summing all 4 weeks.
That problem was successfully solved in b17. Now I've added only StopLoss and ProfitTarget orders and problem is back again. Maybe the solution is similar to the first one?

[NinjaTrader_Jason]

Hello ivanjocic,

It is as expected that when less data and parameters are used, the backtest is performed faster.

I backtested your strategy with 'TotalOrdersPerMonth' set to 1000 and it finished in a few seconds. I left all other parameters (P1 - P10) to zero and I used 5/24/2010 - 6/22/2010.

Could it be you refer to optimizations and not backtests. Note that for each parameter combination, a backtest will be performed. If you use 20;1000;1 for TotalOrdersPerMonth in combination with the other 10 parameters, many backtests will be performed to finish the optimization.

[ivanjocic]

I've developed that strategy for optimization problems but then I noted that the speed (should I say slowlyness) was only related to number of orders generated and not by the parameters and combinations used.
so the problem lays in backtest and you can ignore all of the parameters P1-P10 and try following setups:
1. 5/24/2010 - 6/22/2010 , 10000 orders x month. record timing.
2. use week by week of the same period, leave 10000 orders x month. record timing.

you will see that wk1+wk2+wk3+wk4 generates more or less the same number of orders, but it will take exponentialy less time than executing it on one month

you can also try the same setup but first comment out 2 lines in strategy reffering to stoploss and profit target.
thanks

[NinjaTrader_Jason]

Hello ivanjocic,

When I backtest the strategy on 5/24/2010 - 6/22/2010 with TotalOrdersPerMonth set to 1000 for the ES 09-10, it takes 5-6 seconds to finish the backtest.

When I backtest a week, it finishes in 1-2 seconds.

I can take a little longer when backtesting a month in this case, since historical data for the 06-10 expiry needs to be loaded and merged as well.

[ivanjocic]

Please use 10000 as it's significative value. under 5000 orders more or less everything is fine.

Ivan

[NinjaTrader_Jason]

Ok, using a value of 10,000, it took approximately 10 seconds to finish the backtest on 1 month of historical data.

Backtesting a week took 1-2 seconds.

Could it be you performed the monthly backtest first? So it had to load all historical data, which caused the backtest to run a little longer. Subsequently when you performed a weekly backtest, no historical data had to be loaded from your data feed, since it is already availabe due to the prior monthly backtest?

[ivanjocic]

I new that catch already from last issue I've reported. Maybe something is wrong with my installation, but I see that on Win2k3 server 32bit is performing the same way.
Can you give me some hint on what can be wrong?

[NinjaTrader_Austin]

ivanjocic, there is just more information to compute. Just because the backtest takes longer with three times as many orders to deal with doesn't mean something is "wrong".

In order to be sure there is no data loading to throw off the times, please download the data first and then disconnect from your data provider and then you can backtest to isolate possible speed issues.