Look-ahead bias (and potential bug) in WFO runs

[kbellare]

In my WFO run on minute bars and have 4-day optimization window and 1-day walk-forward test period (total WFO run is 30 days or so).

I noticed that the walk-forward test window results includes trades that happened during the optimization window. This seems to be a serious bug that is introducing a look-ahead bias and inflating walk-forward performance results. This happens for different lengths of optimization and WF test windows. Heres an example -

I collected trades from the Optimization tab (window 6/27-6/30) and from the Walk-Forward tab (test window 7/1-7/1).
- 30% of trades in the Test Window are from 6/30 and these are the more profitable trades - if i exclude these, the trades from 7/1 walk-forward run are collectively unprofitable. The performance reported includes these 30% trades that were conducted during the Optimization window.
- Another strange thing is that the Optimization window also has 20% of its trades on 7/1 (after the window). I'm attaching my spreadsheet with the trades (WF trades and Opt trades tabs), so you can see examples of such trades.
This is confirmed by the fact that my Walk-forward performance on Sunday/Monday is the worst because the markets are closed on Saturday and there are no "biased" trades in the Optimization window to inflate the results on Sunday/Monday.


Question - How do i run a WFO where the Optimization window and Walk-forward window don't overlap and trades are on distinctly separate dates or time periods?

[koganam]

In my WFO run on minute bars and have 4-day optimization window and 1-day walk-forward test period (total WFO run is 30 days or so).

I noticed that the walk-forward test window results includes trades that happened during the optimization window. This seems to be a serious bug that is introducing a look-ahead bias and inflating walk-forward performance results. This happens for different lengths of optimization and WF test windows. Heres an example -

I collected trades from the Optimization tab (window 6/27-6/30) and from the Walk-Forward tab (test window 7/1-7/1).
- 30% of trades in the Test Window are from 6/30 and these are the more profitable trades - if i exclude these, the trades from 7/1 walk-forward run are collectively unprofitable. The performance reported includes these 30% trades that were conducted during the Optimization window.
- Another strange thing is that the Optimization window also has 20% of its trades on 7/1 (after the window). I'm attaching my spreadsheet with the trades (WF trades and Opt trades tabs), so you can see examples of such trades.
This is confirmed by the fact that my Walk-forward performance on Sunday/Monday is the worst because the markets are closed on Saturday and there are no "biased" trades in the Optimization window to inflate the results on Sunday/Monday.


Question - How do i run a WFO where the Optimization window and Walk-forward window don't overlap and trades are on distinctly separate dates or time periods?

Are you testing an instrument whose trading session spans midnight, as for example, most futures?

[kbellare]

Yes, i'm running WFO on a basket of 25 futures - their session runs overnight.
Can i get authentic WFO results by somehow artificially changing the session template to end a few minutes before midnight? Pl advise how ..
Also, if i do so, do i set "Close on End of Session" to False or True in the strategy setting?

thx
Kiran

[NinjaTrader_PatrickH]

Hello Kiran,

Thank you for your response.

On the multiple trades, are you using multiple bar series in your script? May we test your script on our end?

For the WFO item on the overlapping trades, Try setting your Session Template to 'CME US Index Futures RTH' and then running the test. Do you still seeing a "look ahead bias"?

I look forward your response.