In/out of sample testing

[micasa001]

Hi all,

We are working on an automated trading system and now have to integrate the strategy testing and validation part. For this we have the idea to:

1. Select the strategies we have
2. Automate the backtests on different timeframes, pairs, with controle on market types
3. When our required settings are met like min profit factor, max hold time, .. we separate this strategy

We now move to the validation part where we like test the strategy with;

a: monte carlo
b: forward testing

My question now is the we want to prevent overfitting of the results. So:

- How is it possible to do in/out of sample tests in the monte carlo simulations? I dont see much properties in the UI to create randomness in the dataset. How is this possible and how to prevent the overfit?

- Bonus question : you have some suggestions that must need to integrated in the model to validate good probability strategies? (was thinking on robustness testing for introduce testing with delayed entries, exits, different candle lengths etc. But don't know if something like this is in NT8 already - or that plugins are available for this (free/commercial don't mather).

Thx for all your replies and help.

[NinjaTrader_ChrisL]

Hello micasa001, thank you for your post and welcome to the NinjaTrader forum.

The data set for the monte carlo simulation are the trades made from a backtest, so you would need to test a different set of data, gather trade results, then generate another monte carlo normal distribution. A Walk forward optimization will allow you to pick an "in-sample" and "out-sample" data set.

The performance you get from a strategy will depend on the strategy itself. You do have the ability to make your own optimizers and optimization fitness scripts.

Please let me know if I can assist any further.