I'm optimizing around 40 parameters

[AdeptistJune]

Hello,

When running optimization in the strategy analyzer, I'm optimizing around 40 parameters. When doing so, the test uses 100% of my CPU. Does this mean I need more processing power to optimize the script, or is the issue with the syntax of the script?

When running a regular backtest in the strategy analyzer, it only uses 10% of my CPU.

Here are my specs:
Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.60 GHz
32.0 GB (31.8 GB usable)

[UltraNIX]

I highly doubt the efficiency of optimizing "around 40 parameters" all at once. if you want to find "the most perfect of the perfectest" (incorrect grammar intentionally) variation, you should watch a video on OVERFITTING while your computer is using it's powers at optimizing.

What I would suggest instead of upgrading your PC:
1) Test each parameter 1-by-1, using larger steps, so that one optimization has in between 5 and 100 iterations. (And set "Keep maximum # of best results" to your iterations count)
2) Export each optimization set to Excel.
3) Compare and rank which INDIVIDUAL parameters give the biggest impact.

Your first part is done. Now,when you have ranked all individual tests, you can once again test each parameter, starting with the one with most impact and gradually adding all subsequent paramaters until you reach the least impactful parameter, but this time you could use smaller steps, yet I wouldn't go over 1000 iterations, as it sometimes can crash Ninja. And most of the time it's not needed.

I'd also like to suggest you split your current data set to 80-20 proportion, 80% of data for optimization ("in-sample" test), 20% for "out-of-sample" test. And if you see similar results on your in sample and out of sample tests - congrats, you can continue testing on demo for like a month, and if results doesn't bring anything surprising, you are good to go.

If strategy fails to show similar results in in-sample and out-of-sample test, you are probably overfitting and it's not worth effort.

[AdeptistJune]

Hello UltraNix,

Thanks so for the insight. I have a couple of questions:

1) When you say test each parameter 1-by-1, using larger steps: What does using larger steps mean?
2) Can you provide an example of "Compare and rank which INDIVIDUAL parameters"?

[NinjaTrader_BrandonH]

Hello AdeptistJune,

Thanks for your post.

Please see this help guide page for information about best practices for optimizations: https://ninjatrader.com/support/help...ionPerformance

You could use stepping when working with strategies with a large number of parameters. Avoid using "1" as the increment value for the optimizer to avoid forcing the optimizer algorithm to run the maximum number of permutations. Changing the increment value to as little as "2" can cut the number of permutations in half, and increasing this value can have progressively less of an impact.

The Genetic Algorithm could be an alternate option for increasing parameter increment values. Rather than running brute-force tests by iterating over all permutations, the Genetic Algorithm intentionally ignores parameter combinations that are likely to produce sub-optimal results.

Let us know if we may assist further.

[UltraNIX]

Ok, let's take Sample SMA CrossOver strategy.
1) Instead of optimizing both Fast and Slow at once, my idea would be to test Fast with larger steps, like 5;25;5 (i.e. from 5 to 25, by step size of, you'd test 5, 10, 15, 20, 25 = 5 iterations).
2) When you test both Fast and Slow, see which of the paramaters with best results. In my case, I optimize on max net profit. And Slow gave me better results with 20;100;5 optimization. So I would leave Fast for 10 (default) and then best result from Slow was 20, so I'd then go from 10;50;1, meaning, I am lowering step size to one, and testing variations from 10 to 50.

When I'm happy with the result, I then optimize Fast parameter with lowered step to 1 or 2, while keeping optimized Slow parameter. And so on, and so forth.

[JakeOfSpades]

If your optimizing 40 different parameters you are most definitely overfitting your strategy and probably will not perform in the future, if your optimizing 1 parameter with 40 iterations it shouldn't take 100% of your CPU.

---

I build useful software systems for financial markets
Generate automated strategies in NinjaTrader : www.stratgen.io

[AdeptistJune]

Thanks, everyone. The resources provided gave me a framework on how to approach my testing.

[AdeptistJune]

Hello Brandon,

I watched the video on GA. But I don't understand these steps:

1.Begin with an initial population size consisting of randomly selected individuals (parameter setting combinations)

2.Compute the fitness (Optimize on...) for each individual in the population and assign probabilities to the population based on the fitness results. More fit results have more probability in being selected for breeding of the next generation.

3.Generate a new population for the next generation by selecting individuals from the prior generation to produce offspring via crossover and mutation (see below)

4.Repeat from step 2 till you reach the number of generations in your test.

Will I need to manually test each fitness fro each individual?

[NinjaTrader_BrandonH]

Hello AdeptistJune,

Thanks for your note.

These are not steps that you need to take. These steps are how the Genetic Algorithm calculates when you run a Genetic Optimization in the Strategy Analyzer.

From the Genetic Algorithm help guide:

"How the GA calculates

The GA determines its solution through the following steps:

1.Begin with an initial population size consisting of randomly selected individuals (parameter setting combinations)

2.Compute the fitness (Optimize on...) for each individual in the population and assign probabilities to the population based on the fitness results. More fit results have more probability in being selected for breeding of the next generation.

3.Generate a new population for the next generation by selecting individuals from the prior generation to produce offspring via crossover and mutation (see below)

4.Repeat from step 2 till you reach the number of generations in your test"

Genetic Algorithm: https://ninjatrader.com/support/help...neticAlgorithm

The video in the help guide linked above demonstrates how to run a Genetic Optimization in the Strategy Analyzer and explains the different Genetic Optimization Properties available to set.

Let us know if we may assist further.

[AdeptistJune]

Hello Brandan,

I watched the video but I still don't understand the process. I don't understand the sequence of the process. For example: For Fast I pick (5;10;1) and I've set all the genetic algorithm GO properties. Next, I select Run. I see results in the optimization graph. This is step 2, I think. Now, when it comes to step 3 and 4, I have no idea what to do next.

Am I supposed to select something else before selecting Run, again? The video explains the basic functions and a brief explanation of go properties, but it doesn't explain (visually) how to do steps 3 and 4.

Sorry for any inconvience.

[NinjaTrader_BrandonH]

Hello AdeptistJune,

Thanks for your note.

As stated in my previous post. the steps you mentioned are not steps that you need to take to run a Genetic Optimization.

"These are not steps that you need to take. These steps are how the Genetic Algorithm calculates when you run a Genetic Optimization in the Strategy Analyzer."

The steps you mentioned are how the Genetic Algorithm calculates when an optimization is run.

First, the GA begins with an initial population size consisting of randomly selected individuals (parameter setting combinations). Then the algorithm will compute the fitness (Optimize on...) for each individual in the population and assign probabilities to the population based on the results it calculates. The GA will then generate a new population for the next generation by selecting individuals from the prior generation. This is repeated until the number of generations in the test is met.

To run a Genetic Optimization, you would only need to open a New > Strategy Analyzer window, set the 'Backtest type' drop-down menu to 'Optimization', select your strategy, set the 'Optimizer' property to 'Genetic', configure the Genetic Optimization properties you want to test, then click the 'Run' button to run the Genetic Optimization. While the optimization is running, it is using those steps previously mentioned to calculate results.

Let us know if we may assist further.

[Teebone21]

you are optimizing to much man. You gonna overfit and kill your CPU while having a strategy that doesnt work in real time

[AdeptistJune]

you are optimizing to much man. You gonna overfit and kill your CPU while having a strategy that doesnt work in real time

What is overfit, and how do I avoid overfitting?