How does the optimizer work?

[johannes]

Hi,

I can't get it to figure out how the Strategy Analyzer Optimizer works. When I let it run on my strategy, it always comes up with the highest parameters. I let it run between 1 and 10 it comes up with 10, I let it run between 1 and 30 it comes up with 30...while 6 is the best parameter. For the Optimize on.. option I have selected max. net profit. I suppose that with "max. net profit" "Total Net Profit" is meant on the Summary tab?

So can you please help me what I am doing wrong, or where can I find some aditional information about the optimizer (and yes I have read the help page)?

Thanks!

[NinjaTrader_Jason]

If you set 'Optimize on...' to 'Max. net profit', you will request the optimal input values that return the maximum profit.

Please check the 'Keep best # results' in the Optimize parameters window. This setting will determine how many results will be displayed in the Optimizer-tab.

You can click the 'Cumulated Profit' column to sort the results per profit.
Unfortunately there is no additional information regarding Optimzing other than the link below.

http://www.ninjatrader-support.com/HelpGuideV6/OptimizeAStrategy.html

[johannes]

Thanks for your response Jason!

But I still don't get it...

So I've uploaded screenshots of what I'm doing.

The filenames of the pictures have the name that corresponds to the points below.

1) I set the parameters for Set1 and Set 2 to be optimized, and Optimize on "max. net profit" with 30 results.

2) The results after running this strategy optimizer on the NQ. I see Total Net Profit displaying -$100, is that the best result this strategy could have? with the settings Set1=10 and Set2=10?

3) I click the optimizer tab and it displays this information. And all performances have the result -$100???

4) I know the best performance for this strategy is Set1=2 and Set2=6, so I let it run with this, nothing else changed.

5) and the result is $960 net profit but why doesn't the optimizer come up with this value? What am I doing wrong? I really need some help, I would optimize a lot more variables from my strategy, and manual testing costs a lot of time.

Thanks for your help!

[NinjaTrader_Jason]

It seems the custom strategy you use is causing this behavior; set1 and set2 are identical, since each combination has the same result.

However it does not explain why the backtest using set1: 2 and set2: 6 result in a different outcome. If you search the set1: 2 and set2: 6 combination in the Optimizer-tab, does it display a total net profit of $100?

Please perform a optimization using the SampleMACrossover strategy and check if the results make sense.

[NinjaTrader_Jason]

What happens if you use the following setup:

Set1: 1;10;1

Set2: 1;10;1

Set3: 1;2;1

Set4: 1;2;1

[johannes]

It seems the custom strategy you use is causing this behavior; set1 and set2 are identical, since each combination has the same result.

However it does not explain why the backtest using set1: 2 and set2: 6 result in a different outcome. If you search the set1: 2 and set2: 6 combination in the Optimizer-tab, does it display a total net profit of $100?

If I do this then the results are the same as only backtesting with those variables $960 profit.

Please perform a optimization using the SampleMACrossover strategy and check if the results make sense.

When I run this strategy, everything functions as it should. It gives the best and minder good variable values, so it seems that the optimizer if functioning alright.

What am I doing. I've written my own indicator for buying and my own indicator for selling. In my strategy I let the indicator run over different timeframes from 1 minute till 10 minute to find the best timeframes to use my indicator with. Set1 is used for the buying timeframe, and Set2 for the selling timeframe.

[johannes]

What happens if you use the following setup:

Set1: 1;10;1

Set2: 1;10;1

Set3: 1;2;1

Set4: 1;2;1

Only Set1 and Set2 are connected at this moment, Set3 and Set4 aren't used yet in my strategy, so they have no influence on the strategy.

But I still tested it, but didn't make a difference.

[NinjaTrader_JoshP]

johannes,

I have not followed this thread intimately, but you mentioned you are using an indicator you made in your strategy. By chance, do you have a CalculateOnBarClose line in your indicator? If you do you will need to remove this line from the indicator.

[johannes]

Thanks for your response Josh!

Just removed the CalculateOnBarClose lines in my indicators, still have a line in the strategy.

Okey, now the parameters in the Optimizer don't display a countdown from 10/1 till 9/1 and so on anymore. But I ran one backtest with the settings 2/6 before the Performance column gave still 930, when I clicked different lines, I saw at the periods tab that different profits where reached. After going through the results I found out that the setting 8/3 had a profit of $1530. So I ran a backtest with that settings and indeed it worked out. Then did a optimizer again, and the last backtest result keeps staying in the performance column as you can see in the screenshot I added.

So it is still not running as it supposed to!

What are other things I have to think about, like the CalculateOnBarClose lines?

[NinjaTrader_JoshP]

johannes,

So the problem you have is the second optimize produces static results?

[johannes]

No.

When I start from the Control Center a new "Strategy Analyzer" then select "Optimize" and then set my parameters 1..10 for Set1 and Set2, as "Total Net" it returns $4035 When I go to the "Optimizer Tab" then it says on each row of the "Total Net" column $4035, but when I click on a row and look at the Periods tab I see whole other results.

So the question is "what is screwing the Optimizer up"?

[NinjaTrader_JoshP]

The Periods tab is not directly comparable to the Total Net column though. You need to view the Summary tab for the Total Net Profit row.

[johannes]

The Periods tab is not directly comparable to the Total Net column though. You need to view the Summary tab for the Total Net Profit row.

Summery gives for each row of the results of the optimizer the same result.

But when I set Periods tab to "Annual" it sums all results and gives the Total Net Profit of that strategy, because when I backtest the settings, the results are the same as the "Annual" on the Periods tab, and are NOT the same as the numbers on the Summeray tab.

There is probably some code in my strategy or indicator that is fu..ing things up, but what can that be?

[NinjaTrader_JoshP]

johannes,

I do not know. I suggest you simplify your code as much as possible and try to find the breaking point. One thing that may or may not have influence is if you try to define variables in the Initialize() method. You cannot do that there if you are doing that at all.

[johannes]

This is how my initialize looks like:

[FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff]
[SIZE=2][FONT=Courier New][COLOR=#0000ff]protected [/COLOR][/FONT][/SIZE][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff]override [/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff]void[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][FONT=Courier New][SIZE=2] Initialize()[/SIZE][/FONT]
[SIZE=2][FONT=Courier New]{[/FONT][/SIZE]
[SIZE=2][FONT=Courier New]CalculateOnBarClose = [/FONT][/SIZE][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff][FONT=Courier New][SIZE=2][COLOR=#0000ff]true[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][FONT=Courier New][SIZE=2];[/SIZE][/FONT]
[SIZE=2][FONT=Courier New]Add(PeriodType.Minute, Set1); [/FONT][/SIZE][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000]//2[/COLOR][/SIZE][/FONT]
[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][FONT=Courier New][SIZE=2]Add(PeriodType.Minute, [/SIZE][/FONT][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#800080][FONT=Courier New][SIZE=2][COLOR=#800080][FONT=Courier New][SIZE=2][COLOR=#800080]9[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][FONT=Courier New][SIZE=2]);[/SIZE][/FONT]
[SIZE=2][FONT=Courier New]Add(PeriodType.Minute, Set2); [/FONT][/SIZE][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000]//6[/COLOR][/SIZE][/FONT]
[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000][FONT=Courier New][SIZE=2][COLOR=#008000]//SetStopLoss(CalculationMode.Ticks, 28);[/COLOR][/SIZE][/FONT]
[/COLOR][/SIZE][/FONT][/COLOR][/SIZE][/FONT][FONT=Courier New][SIZE=2][FONT=Courier New][SIZE=2]}[/SIZE][/FONT]
[/SIZE][/FONT]

And I'm trying all kind of things, but can't figure it out yet...