Getting started w/optimization

[bluelou]

I'm new to optimization w/NT and I'm looking to use the default and genetic optimizers on some existing custom strategies. I have 3 questions in getting started:

1) Do all parameters to be optimized need to be declared as type int?

2) For get/set must parameters first be declared as private (i.e., fast) and then in Properties as public (i.e., Fast) as is done in the SampleMACrossOver strategy?

3) If a strategy has 10 parameters but I only want to optimize 5 of them is it required that only the 5 to be optimized are included in Properties? Are there any other considerations when combining parameters that are to be optimized w/params that will not be optimized?

Thx,
Lou

[NinjaTrader_Brett]

Hello,

Thank you for your forum post.

1) They would need to be a double or int to optimize them.

2) It is best to replicate the SampleMACrossOver in this reguard with parameters.

3) You can select what parameters you want to be optimized, by default unless you change the optimize range for a parameter when you run a optimize run it will run the same as a backtest with default parameters.

Let me know if I can be of further assistance.

[bluelou]

Hello,

Thank you for your forum post.

1) They would need to be a double or int to optimize them.

2) It is best to replicate the SampleMACrossOver in this reguard with parameters.

3) You can select what parameters you want to be optimized, by default unless you change the optimize range for a parameter when you run a optimize run it will run the same as a backtest with default parameters.

Let me know if I can be of further assistance.

Re: 3) How do I select what parameters I want optimized (you mentioned this in 3) above) ?

FWIW, my plan is to get started w/the SampleMACrossOver strategy and then add a few parameters to it until I get comfortable enough with the features to try it on my own strategy.

[NinjaTrader_Brett]

Hello,

You would select this using the following instructions when starting the optimization run.

http://www.ninjatrader.com/support/helpGuides/nt7/optimize_a_strategy.htm

Let me know if I can be of further assistance.

[bluelou]

Hello,

You would select this using the following instructions when starting the optimization run.

http://www.ninjatrader.com/support/helpGuides/nt7/optimize_a_strategy.htm

Let me know if I can be of further assistance.

Re: 3) How do I select what parameters I want optimized (you mentioned this in 3) above) ?

I just re-read the link you posted. I didn't see anything there about how to select specific parameters for optimization. Could you be more specific? on how to do this or where this info is located?

[NinjaTrader_RyanM1]

Hello bluelou,

You can optimize public properties that are numeric values. Correct - SampleMACrossover offers examples of public properties. These are Fast and Slow periods.

We document how these are setup as well here:
Creating User Defined Input Parameters

There's no selection of parameters - any numeric public input can be optimized. You specify Min, Max and Increment to optimize a specific input.

[bluelou]

I suspected that was the case but I wasn't sure. Thx.

[bluelou]

I've been able to get both the default and GO optimizers to run on my strategy but I'm only getting 2 optimization test runs to display with each optimization. One test usually shows zero as the "performance" result and the other is the final result. This is occurring despite setting "Keep best # results" = 10.

How do I get the 10 best results to display?

[NinjaTrader_JoshP]

Is your strategy running into any errors? Please try with SampleMACrossOver to confirm behavior first.

[bluelou]

Is your strategy running into any errors? Please try with SampleMACrossOver to confirm behavior first.

No, no errors. SampleMACrossOver displays top 10 results as expected. I can't figure out why I'm getting only 2 results displayed in my custom strategy. Any ideas? How do I know that it's even doing the full search?

[NinjaTrader_JoshP]

bluelou,

The Default optimizer is a brute force optimizer so it will go through all permutations set out by your parameter settings. The Genetic one will not go through all permutations since that is the nature of Genetic.

You will need to check into the code of your particular strategy to see what is happening under the hood. I suggest you just add a one line print when CurrentBar == 0 or something like that with the parameter settings being tested then reading your output window afterwards.

[bluelou]

bluelou,

The Default optimizer is a brute force optimizer so it will go through all permutations set out by your parameter settings. The Genetic one will not go through all permutations since that is the nature of Genetic.

You will need to check into the code of your particular strategy to see what is happening under the hood. I suggest you just add a one line print when CurrentBar == 0 or something like that with the parameter settings being tested then reading your output window afterwards.

What am I looking for? You're saying to output the parameter settings to a log file? Will the optimizer show different values being tested on each new bar? Is that what you're suggesting.

I understand that brute force searches all permutation while GO does not. The strategy is running properly so I'm not sure what to isolate and put in a log file. Could you be more specific regarding the test you're suggesting?

[NinjaTrader_JoshP]

If you are processing different parameter sets, you will get different print outs of the different parameter settings the strategy is cycling through as it optimizes. That is how you will know if you are actually testing all the combinations you have set out to test.

[bluelou]

Okay, that's what I was asking. Thx.

[bluelou]

Unclear how to interpret Walk-Forward Optimization results.

After running a WFO, I see the individual out-of-sample (OOS) test results as I would expect.

The WFO also generates a report over the full time frame tested. The summary tab doesn't match the individual trades, the trades don't match either the in-sample or OOS periods, and the parameter settings fall outside the min/max ranges that I specified. In fact, the parameter settings don't appear to be optimized at all, they're just the default parameter settings in the strategy. Is this the expected result from a WFO test?