WalkForward issue...

[kenb2004]

[email protected]You get it! It's broken and evidently always has been. But not enough users have complained for Ninja to deal with the problem. They want you to complain. "We do not get enough reports of this to focus efforts in changing this in NinjaTader 7" is what we been told. This just amazes me that an issue this critical to the end process of automated trading strategy development that NinjaTrader would NEED to be told by the USERs that this problem needs to be fixed.

And this is not the ONLY issue with WFO. This just blew me away! So as not to muddy the already muddy waters, if you use "Instrument settings" for ES with a 4:1 Opt/Period:Test/Period, not only will WFO include the weekend days, it also includes the TestPeriod day in the Opt/Period and the results are always REALLY GOOD! This is because WFO does not recognize days of the week. The Instrument settings for ES uses CME US Index Futures ETH, which includes 2 "DAYS" of trading, 5pm from the previous day until 415pm of the trading day. So when you go back and look at the optimization periods trades, sure enough the Test Period trades are there as well. You can see them right on the chart with the test period date at the top.

I would say, "what a joke", but it's no joke when your money is envolved.

[Camdo]

2.Trades within the test results overlap, so the total cummulative profit for the WFO period is erroneous. Thus one does not know if the optimized parameters would be effective as an adaptive strategy. - I do not see why this is an issue. They would overlap, as you are testing/optimizing on a rolling basis, so the last day of the WF period becomes the last day of optimization for the next period's trades. This actually looks correct to me, but I am willing to be corrected.

My purpose for using WFO is to compare profitability of a strategy with static parameters over a given time frame vs adapting parameters to changing market conditions. For example: At the start of a new day, run an optimization using the prior 2 days as timeframe. Run the strategy with the optimal parameters for the day. Repeat the method each day. Would this yield superior profit over the course of six months than just holding the parameters static?

The problem is WFO cannot deal with the overlapping trades. In some instances the trade is truncated "exit on close" at the end of the test period and in other instances the trade is extended prior to the start of the trading period. When trades overlap, there are twice as many shares in play. When trades are truncated, the position is lost and without an entry signal in the next test period. So the cumulative profit over many test periods is entirely erroneous. Thus there cannot be any profit comparison using static parameters over the six month timeframe.

I don't know for what purpose WFO, as NT has presented it, would be used. For my purposes, there needs to be a "merge" option so the trades are continous across test periods. If trading live, I would hold a position until the strategy exited before updating to the next optimal parameters.

[koganam]

My purpose for using WFO is to compare profitability of a strategy with static parameters over a given time frame vs adapting parameters to changing market conditions. For example: At the start of a new day, run an optimization using the prior 2 days as timeframe. Run the strategy with the optimal parameters for the day. Repeat the method each day. Would this yield superior profit over the course of six months than just holding the parameters static?

The problem is WFO cannot deal with the overlapping trades. In some instances the trade is truncated "exit on close" at the end of the test period and in other instances the trade is extended prior to the start of the trading period. When trades overlap, there are twice as many shares in play. When trades are truncated, the position is lost and without an entry signal in the next test period. So the cumulative profit over many test periods is entirely erroneous. Thus there cannot be any profit comparison using static parameters over the six month timeframe.

I don't know for what purpose WFO, as NT has presented it, would be used. For my purposes, there needs to be a "merge" option so the trades are continous across test periods. If trading live, I would hold a position until the strategy exited before updating to the next optimal parameters.

I believe what you describe is the way the WFO works right now, but I can see why there might be data bounds problems when session times may overlap days, and the day measure is not synchronized to the session measure.

Well, I really have not seen this particular issue, simply because I always use RTH templates, as, other than stock options, I never hold anything overnight, so trading periods are matched to days for me.

I certainly hope that they fix all this in NT8.

[Camdo]

I see now that overlapping trades is problematic with the session template chosen.
Using ES futures which trade 24 hours a day except weekends. If session template "24/7 default" is used, then trades will overlap. If "CME Index Futures RTH" is used, then trades do not overlap. The problem is "24/7 default" is needed for 24 hour trading, RTH is approx. 9:30AM to 4:00pm.

Test period within the optimize period is also fixed using RTH rather than 24/7 (or ETH).

It seems that NT7 has a bug in it that it cannot read 24 hour time of day.

[kenb2004]

WFO Fixes Wish List:

1. Recognize Days of the Week
2. Option to turn off certain dates, like holidays
3. Option to turn off certain days of the week, like Mondays, or Weekends
4. Option to optimize Optimization Period
5. Option to optimize Test Period

Please add to this list

[Camdo]

I made a custom Session Template for ES futures trading using extended trading hours "MyCmeETH" see attached:

This template solves the problem of overlapping trades and test period within the optimize period.

"Exit on close" at the test period boundary remains a problem for adaptive strategy backtesting.

I would add to the above WFO wish list:
6. Option to extend test period until flat.

[NinjaTrader_Matthew]

Thank you all for providing your feedback in this area, and we appreciate the workarounds that have been suggested.