Trades collections in SystemPerformance sometimes empty during optimization

[philcollins]

I'm using a custom OptimizationType implementation to evaluate the performance of my systems during optimizations. However, I can't seem to get the various TradeCollection objects to work correctly, they are empty 1/3 of the time (have checked in the debugger).

As a simple test try to use this as an optimization type implementation:

public override double GetPerformanceValue(SystemPerformance systemPerformance) {
			this.Strategy.Log("Count: " + systemPerformance.AllTrades.Count, NinjaTrader.Cbi.LogLevel.Information);
			return systemPerformance.AllTrades.Count;
}

You'll see that the entries in the log are correct 1/3 of the time and are '0' 2/3 of the time, and that the performance score in the result list is always 0.

If you use the debugger you'll see the collections like AllTrades, LongTrades etc. are empty while the performance statistics are still available. So using systemPerformance.AllTrades.TradesCount will work, but systemPerformance.AllTrades.Count will just return '0'.

Does anyone know what's going on here?

[NinjaTrader_PatrickH]

Hello philcollins,

Thank you for your post and welcome to the NinjaTrader Support Forum!

You should find that Performance will report the proper values for Count and TradesCount.

[NinjaTrader_PatrickH]

Here is what I tested:

			Print("");
			Print(Time[0]);
			Print("Count: " + Performance.AllTrades.Count);
			Print("TCount : " + Performance.AllTrades.TradesCount);

I believe you are testing SystemPerformance which is an internal item that was not meant to be called and thus may produce unexpected values.

[philcollins]

It seems the Performance property is only available on the Strategy class though, not on OptimizationType. I managed to access it via this.Strategy.Performance, but this yields the same erroneous results unfortunately.

[NinjaTrader_PatrickH]

Hello philcollins,

Thank you for your response.

Do you have a script we can test to see what is occurring?

[philcollins]

Alright found out what the bug is, it isn't in the script per se, it's in the 'Aggregated' setting of the optimization properties.

If you use a strategy to optimize a basket of instruments with the 'Aggregated' property set to 'True', and then switch back to optimizing just one instrument, the collections will be empty like I described above.

Note that the 'Aggregated' property disappears when you optimize only one instrument.

So I guess the workaround is to set 'Aggregated' to 'False' before optimizing a single instrument.

[NinjaTrader_PatrickH]

Hello philcollins,

Thank you for your response.

May we test the script you are using?

[philcollins]

It doesn't matter which strategy you use really, just use SampleMACrossOver for example.

Your OptimizationType implementation should return e.g:

return systemPerformance.AllTrades.Count;

Right click a basket of instruments, click optimize, set the Aggregated property to True. Click a single instrument and run the optimization. You'll see the OptimizationType will always assign a score of '0' to a run because the AllTrades collection is empty.

To fix it you go back to test a basket and set Aggregated back to False.

[NinjaTrader_PatrickH]

Hello philcollins,

Thank you for your response.

I was hoping to see the Optimization Type you developed or a version that re-creates this as well.

[philcollins]

It's just the one line in my previous post.

[NinjaTrader_ChelseaB]

Hello philcollins,

May I confirm that you have made a copy of the DefaultOptimizationMethod class and that you have added that one line to that copied class?

Are you using an optimization class that is empty without any code in at all including methods?

In what method did you add this line of code?

May we have the optimization script that you have written?

[philcollins]

The initial script I created was a bit more complicated than just this line. But the simplest script you can reproduce the problem with is indeed a copy of one of the standard OptimizationType implementations with just the line I posted above as the function body.

I'd post a complete sample but I'm away from my computer for the next 2 weeks.

The main problem is in using the trades collections, so any scripts that use those collections should exhibit this erroneous behavior.

[NinjaTrader_Jesse]

Hello,

Thank you for the reply.

I was able to test this and I do see that specifically during a basket test with Aggregated set to true, the optimization type reports 0 for the AllTrades and after switching from a basket to a single instrument it seems to retain the aggregated setting.

I went further to test this in NinjaTrader 8 and it has been resolved for NinjaTrader 8. I will submit this to development for review but because this has already been addressed in NT8 it will likely not be resolved for NinjaTrader 7.

I look forward to being of further assistance.