Curve Fit vs Good Strategy

[shortorlong]

When is a strategy that is profitable just a curve-fitted result, vs a genuinely good strategy?

What are some metrics from the strategy analyzer that I should be looking at to determine that my strategy profit is not just a fluke?

What are some conditions you have on the performance of your strategies before you will stamp a strategy as properly tested, and properly profitable?


Really appreciate any good dialogue on this important topic.

[NinjaTrader_JoshP]

Well the nature of backtesting is that it is generally curve-fitting. This is why many system developers insist on forward testing as well. You never know how your strategy truly performs unless you forward test it than you have valuable information as to how your strategy would play out.

[shortorlong]

i see, so there arent certain profit factors, or sharpe ratios or drawdown statistics that would give insight on whether your strategy was better than a coin flip or not eh?

[NinjaTrader_JoshP]

Those ratios are definitely useful but I am not aware of any set rule as to what is good and what is bad. It really just comes down to interpretation and how much risk you are willing to take.

[LG]

When is a strategy that is profitable just a curve-fitted result, vs a genuinely good strategy?
Really appreciate any good dialogue on this important topic.

Great article on this topic (it's complicated, but well worth poring over) in Curtis Faith's blog, called The Optimization Paradox.

http://www.tradingblox.com/tradingblox/optimization_paradox.htm