RC1 strategy analyzer optimization results: some rows zero PnL, zero trade count

[adamus]

Using RC1 for optimizations, I occasionally see a row with results as if there were zero trades.

See attached image.

When I click on the incorrect row and launch the performance viewer, the summary tab shows all zero entries. If I change the display type from Percent to Currency, the correct figures and data all appear. If I change it back to Percent, the correct data remains.

Have you already registered this issue? I have seen it about 10 times today.

[NinjaTrader_RyanM1]

Hello Adamus,

I'm not sure why you're seeing this. Please try with SampleMACrossover strategy and let us know if you can isolate the steps needed to see it.

[adamus]

Hi Ryan,

I'm unable to reproduce this with the SampleMACrossover strategy.

It's not consistently reproducible, but it occurs often enough to mess up my work when I don't see it immediately.

[NinjaTrader_RyanM1]

Thanks, adamus. Please let us know if you have a simple strategy and steps needed to reproduce this.

[adamus]

Yes I do. If you give me your email address I can email it directly to you.

[NinjaTrader_RyanM1]

Please send to [email protected] Attn: 393870

[adamus]

Looking more closely, I see that the results vary from backtest to backtest for the same code, parameters and dates.

This would explain why sometimes one of the backtests in the portfolio result set is zero, but when I look at it in the performance viewer trades or chart tab, because NT reruns the strategy, a result set with trades appears.

I don't understand how the strategy I am running can produce varying results for the same test parameters.

The only thing that occurs to me is that it could have something to do with the MinimumBarsRequired parameter - perhaps I have set it too small, or perhaps I should actually work out whether I really have enough bars of historical data instead of making a crude estimate.

But why would that cause the results to vary from run to run?

[NinjaTrader_RyanM1]

adamus,

What is different about the two tests you are running? Are you running one in strategy analyzer and the other in a chart?

[adamus]

No, they are any consecutive runs of the same strategy over the same instrument and period with the same parameters, in the strategy analyzer.

Of 20 instruments in the portfolio, at least one instrument is always zero PnL and zero trades, and 3, 4 or 5 instruments' results are different to the previous run.

[NinjaTrader_RyanM1]

Thanks. I've not yet received the strategy. I will test and let you know what I find out once it arrives.

[adamus]

Actually I think I might have fixed it, just after sending it to you.

I'll check it out further and see if it's what I think.

[adamus]

Worked out what causes it, don't know why

I've got this code in my onBarUpdate to prevent any trading until all the indicators are initialized, because sometimes I get the dates of the historical data available wrong:

        protected override void OnBarUpdate()
        {
            if (BarsInProgress == 0 
                && pivot.R1.ContainsValue(0)
                && ma1.Value.ContainsValue(0)
                && volatility.Value.ContainsValue(0)
                && atr1.Value.ContainsValue(0))

But it causes the start of trading to vary seemingly randomly from one backtest to another.

In the description of ContainsValue it specifically says

Checking for Valid Values
It is possible that you may use a DataSeries object but decide not to set a value for a specific bar. Internally, a dummy value does exist however, if you wanted to check to see if it was a valid value that you set, you can check the following method.

DataSeries.ContainsValue(intbarsAgo)
Returns a true or false value.

So how am I meant to use it if my code above isn't right?

[adamus]

There is still something wrong.

It's not just that ContainsValue code. There is something else wrong still. Every 1 in 10 runs, I get a different result. It's not as bad as the initial problem where I was seeing zero PnL and zero trades, but it is definitely a different result.

The start date is always the same so I suspect that since I am not checking the ContainsValue on the indicator, one of them is not initialized properly and leads to random results.

[NinjaTrader_RyanM1]

Please try with those contains value checks removed to see if it's contributing here. Are your results then consistent?

You should also check the values of your indicators to make sure they're what you expect.

[adamus]

another question

I have a question related to my current situation regarding the start date of the historical data available in the Historical Data Manager and the start date when onBarUpdate() is first called in the backtest.

I have the From date set to 2009-03-01 but for this instrument I only have data from 2009-10-14.

I set BarsRequired to 100 on a 5 minute timeframe, i.e. less than a day.

In my Print output, I see that the first date printed is 2009-11-13.

Why controls this? Surely it should be 2009-10-14?

This is my code at the start of onBarUpdate():

        protected override void OnBarUpdate()
        {
            if (emergencySituation()) return;
            if (BarsInProgress == primaryTF 
//                && pivot.R1.ContainsValue(0)
//                && ma1.Value.ContainsValue(0)
//                && volatility.Value.ContainsValue(0)
//                && atr1.Value.ContainsValue(0)
                )
            {
                Print(Time[0].ToString("yyyy-MM-dd HH:mm:ss")
                    + " pivot.R:" + pivot.R1[0].ToString(formatPattern(TickSize))
                    + " pivot.S:" + pivot.S1[0].ToString(formatPattern(TickSize))
                    + " ma1:" + ma1[0].ToString(formatPattern(TickSize)) 
                    + " volatility:" + volatility[0].ToString(formatPattern(TickSize)) 
                    + " atr1:" + atr1[0].ToString(formatPattern(TickSize)));
                if (pivot.R1[0] > 2 || pivot.R1[0] < 0.1 || 
                    ma1[0] > 2 || ma1[0] < 0.1 || 
                    volatility[0] > 1 || volatility[0] < 0.00001 || 
                    atr1[0] > 1 || atr1[0] < 0.00001)
                {
                    Log("out of bounds", LogLevel.Alert);
                    emergency = true;
                }

What I am seeing is that the Pivot indicator sometimes produces zero values for that date which causes the problem I'm having.

Normally it's like this on the first bar:


2009-11-13 18:00:00 pivot.R:0.94448 pivot.S:0.93428 ma1:0.97428 volatility:0.00030 atr1:0.00443

But once in a while, randomly, about 1 in 10 or 1 in 20 runs, I get this:

2009-11-13 18:00:00 pivot.R:0.00000 pivot.S:0.00000 ma1:0.97428 volatility:0.00030 atr1:0.00443

If I check for Pivot.R1.ContainsValue(0) and it's false, then my strategy won't start on that bar, so the run will have a different number of bars in the run compared to runs where Pivot.R1 behaves.

What is the situation? Has anybody else raised bugs against the Pivot indicator?