Different parameters for different instruments in a backtest on a list

[adamus]

Hi

I have a strategy with 4 optimizable parameters, and I have a list of instruments - forex pairs.

I have optimized each instrument individually so it has its own best values for the optimizable parameters, now I want to see what the accumulated results are for a backtest using all the instruments with their individually optimal parameter values.

Is this possible?

I am currently looking at the result of optimizing the complete llist of instruments, and I'm asking myself this:

(1) is NT optimizing the whole list of instruments to obtain an optimal value for the parameter across all instruments?

or

(2) is NT finding the optimal values for each parameter for each instrument, and then combining the results at the optimal parameter values, i.e with different values for a parameter on different instruments?

In fact I optimize the strategy once for long and again for short trades. I have a parameter where 1 means 'trade long only' and -1 means 'trade short only'. I intend to trade both at the same time - it's not either long or short.

Ideally I would be able to add each instrument to the list twice and apply the strategy with different optimizable parameters, and the 'long-short' parameter to tell the strategy if it's trading long or short.

Is that possible?

[NinjaTrader_JoshP]

1. You can think of each instrument as independent of the other for the tests.

2. Each optimization is for each parameter on an instrument basis. The COMBINED RESULTS column is all the best results for each instrument combined together.

Unfortunately you cannot add the same instrument twice. For such a procedure I suggest just opening two Strategy Analyzers and running one for your longs and one for your shorts.

[adamus]

Sounds good. I was worried it would be one optimization across the whole list.

I do need to see what happens when I combine the long strategy and the short strategy though. Will I have to export the trades list for each instrument into excel to do that? In fact, will I have to export the trades instrument by instrument? It doesn't look like I can get at the trades for the "combined" list.

In fact it looks like there are only three analysis tabs for the "combined" results - "Summary" which is obvious enough, "Charts" which isn't working (maybe I'm not giving it enough time to load - does it load all the combined trades on a chart?) and "Settings" which shows the values for the parameter that I optimized in a light green cell, for reasons that aren't clear. Can you say what that means?

I need to check correlations of individual instruments' trades as well, to find out if there are big days in there when all trades go against me, i.e. what the biggest losing day was.

[NinjaTrader_JoshP]

adamus,

You will need to export and combine in Excel. Or you can just roll your strategy into one master strategy that contains both longs and shorts.

The COMBINED RESULT row will only have those tabs available since it does not persist details of all the various instruments at once.

The light green cell color means that those are the parameters being optimized and it will show you the optimized value.

[adamus]

Thanks for the info.

Unfortunately the strategy works best when the long and the short trades are managed seperately. When done simultaneously, it makes the optimization process really difficult. I could make a master strategy for both longs and shorts and only use it for live trading, but that leads on to another issue - I would probably have to hard code the parameter values.

I don't actually optimize all four parameters at the same time, that would take days to test all the combinations. I optimize the first 2 parameters, and then use the best parameter values to optimize the second 2 parameters. This means I can finish optimization in minutes. However with NT, it seems I can't do this.

When I have optimized the first 2 parameters, I cannot use the best values for each instrument - I have to set each parameter to one value across all the instruments in the basket in order to run the optimization on the 3rd and 4th parameters.

Is there a solution to this which I've missed?

Thanks

[NinjaTrader_JoshP]

No, you will need to just need to run different Strategy Analyzers to continue optimizing with different values on different instruments.

[adamus]

I guess I would prefer to hard-code the different parameter values for different instruments so that I can carry on with everything in one Strategy Analyzer, to keep the advantages of seeing a combined result.

Is there a way I can do something like this:

if (instrument.symbol == 'USDCHF')
    param = 1.6;
else if (instrument.symbol == 'EURUSD')
    param = 0.8;
etc 
etc

[NinjaTrader_JoshP]

adamus,

You can use Instrument.FullName to check the name of the instrument.

[adamus]

Great, that should be useful.

Another question - probably not implemented since I can't see it in the Help - how can I stop an optimization / backtest / trading, using a Ninja Script function?

If I program in these param values for each instrument, I want to halt NT if I mistakenly try to run the Strategy on an instrument I haven't program the param value for, i.e.

if (instrument.FullName == "USDJPY")
   param = 1.1;
else
   Error("instrument " + instrument.FullName + " not recognized!");

[NinjaTrader_JoshP]

There is no stopping of a strategy run, but you can make it skip all strategy logic though. Please see here: http://www.ninjatrader-support2.com/...ead.php?t=4804

[adamus]

I see that will stop trading, although it won't stop an optimization run or a trading session.

My point is that I would like to avoid leaving an optimization to run overnight and then discover I had wasted my time, or to set up NT to start trading, and then discover much later that it wasn't doing anything.

Admittedly I could add some sort of alert, but I think if I was running an optimization, wouldn't the alert be fired thousands of time very quickly as the Strategy Analyzer ran?

I would like to put forward this requirement as something to be considered for the next release.

Thanks.

[NinjaTrader_JoshP]

adamus,

Similar to stopping trading, you can stop processing of any meaningful strategy logic effectively returning out of OnBarUpdate on every single bar. When you are just coming out of it you should be able to finish an optimization in a matter of seconds-minutes because you simply aren't doing anything on it, just cycling through the bars.

[adamus]

What I am now doing is scripting my strategy with some of the optimized parameters hard-coded in the script.

This means that I can optimize my 4 parameters, but I don't have to do a huge batch run optimizing all possible parameter values against each other - which would take days.

I run my first optimization on one parameter and then I want to hard-code the optimal value for that parameter, but I need to know which event to put the "param1 = 9.876".

When I set up the optimization for a list of instruments in the Strategy Analyzer, I edit the parameter value to 1 or 0 or whatever, doesn't matter, both start and end value. So then my script must set the parameter to the optimal value for each instrument.

However, which event do I script it for?

For instance, with optimizable param1, if I script it with a value in Initialize(), the setParam1() overwrites it. This surprised me. Is there another event that is fired after the Strategy Analyzer sets the parameters at the beginning of each run, before the first bar is called?

[NinjaTrader_JoshP]

Sorry I am not following you. There is no "event" for optimization. You either have a hard set value in for your parameters for things you do not want in your optimization or you have a range set for it which would then be utilized in the optimization. There is no dynamic setting of this in the script. You either have the parameters optimizable from the start or not.

[adamus]

Yes you are right, I appreciate that. I'll try to explain it differently.

When my strategy is run during the optimization process on a list of instruments, then NinjaTrader moves through the instruments and optimization values.

For instance I have $EURUSD, $GBPUSD and $USDCAD, and my strategy has parameters param1, param2 param3 and param4 for optimization.

As NinjaTrader processes the optimization batch, each time it changes an instrument or optimization value, I assume that NinjaTrader calls the SetParam1(), SetParam2(), SetParam3() and SetParam4() methods on my strategy, and the Initialize() method too.

Is that correct so far?

I would like to know which way round it happens. Is it the SetParamX() first and then Initialize(), or vice-versa?