Back testing VS Walk forward paramater trial and error questions.

[olingerc]

Here is some general background information. I have been studying the markets extensivly for 2 years and the past 6 months the emini ES. I have developed a automatic trading strategy and during a back test it yields 9,467 after commissions per contract traded since Jan 1st, 2011. When I run a walk forward test with the same strategy and with default settings it yields 7,213 after commissions per contract. Should I be adjusting any of the "walk forward" settings to better test out my strategy?

Reading here on the forums I have come across strong advocates for walk forward testing verses straight back testing exclusively. The way I "optimized" my strategy was trial and error. Adjust 1 thing and back test it against the same data series and see if the profit was more or less and after a while I came up with the "optimal" settings for the time frame Jan 1st to present. Should I be adjusting my strategy parameters to yield the most profit from a walk forward test instead of a back test? Also what settings if any should I change from default for a walk forward test. Anyone experienced or anyone else that can add anything is greatly appreciated.

I have started the strategy on a live account. I do not have sufficient data with a live account as of yet, but only time will tell. To me my results seem rock solid both in a back test and walk forward and general market directly does not matter because it is a intraday system. If I need to "come down from a fantasy" please someone chime in.

If it makes any difference no matter what paramater I change in my system it does not ever yield more than 7,213 for a walk forward, small tweaks only make it yield less. Is this strategy already "maxed out" so to speak?

[NinjaTrader_JoshP]

olingerc,

A standard backtest only suffers greatly from the pitfalls of curve fitting. You end up optimizing to match the curve, but that is not necessarily characteristic of how your strategy would perform going forward.

The reason people use walk forward optimization is to avoid such curve fitting issues and generate results for each test period based on previous period's optimizations. This means you are taking your optimized out values, pushing it forward, and getting that performance. This greatly reduces the curve fitting nature of just a straight backtest.

You should also always take your results with a grain of salt. It is just a basis point to have an idea about how your strategy could have performed. If you can't get any higher than 7213, then that is likely the highest it will go. Instead of focusing on tweaking little knobs here and there, it would likely be more beneficial to tweak the actual logic and algorithm of the strategy to improve it.

[olingerc]

Thanks Josh. I know it is very "unfair" to manipulate parameters to fit past events. Having 20/20 hindsight is a useless super power when it comes to the uncertainty of future events, this I do realize. Walk forward algorithms from what I understand is suppose to give a more "realistic" performance of a strategy correct? I just found it odd that I could not improve my Walking forward profit at all with any tweaking.

Since Jan 1st both walk forward and back test show around 40% profitable trades to losing trades and with a average winning trade almost double the average losing trade. It is hard for me to believe that this would go flying out the window with a live account. I guess my live account bank roll will be the final test. Ive run the live account strategy in tandem along with a sim account the fills seem to be at the exact time and price so I don't think real time slippage will be a big factor.