Strategy backtest "Combined Results" bugs

[kdoren]

The "Combined Results" line when backtesting an instrument list incorrectly calculates the "Performance" (same as "Profit Factor") in NT7 beta 3.

The correct calculation is (Combined Gross Profit) / -(Combined Gross Loss)

NT7 (and probably also NT6.5) appears to use an average of the profit factors for each instrument weighted by the number of trades for that instrument. This is very wrong, because an instrument can get a big profit factor even while making very little profit (i.e. a single tiny winner gets a profit factor of 99) which will swamp a much more profitable instrument with a more trades, some of which are losers.

This is a big deal now that NT7 allows a strategy to be optimized for combined profit factor, which is a very useful thing to be able to do, but worse than useless with the current calculation. I saw one example where NT7 reported a combined profit factor of 8.1 when the correct number was 1.4.

This is less important, but the combined result for "Max Drawdown" is pretty useless, I expect it is also some kind of weighted average. The only really useful number would be to calculate the maximum account drawdown if all instruments were traded simultaneously, but I expect this would be difficult to implement.

Also, I believe that in NT6.5 I could select for backtesting a subset of an instrument list by shift-click or ctrl-click on the intruments. This doesn't work in NT7, I can only select the entire list if I want to backtest more than one instrument. Please fix this if possible.

Thanks!

[NinjaTrader_Dierk]

- Profit factor: we'll change that with next update. Thanks for making us aware
- DrawDown is a weighted average. This is by design.
- multi-select on the strategy analyzer no longer is available since there had been conceptual inconsistencies in 6.5

[dave1992]

Hello,

The trades per day is calculated incorrectly in the combined results (it's much too small)

Also, and this is not a bug, but it would be really really nice to implement the graphs tab for combined results. I currently have to export the trades from each result test and do much messing around with data just to see profit per month etc. I imagine most people using combined results will be planning to trade multiple instruments at the same time.

thanks
Dave

[NinjaTrader_Dierk]

This is a bug which will be fixed with next update. Thanks for reporting.

[pmorissette]

Can you please make it such that we can see the same stats (tabs) in the overall entry as the ones in the individual securities? The information seems to be there. At the very least, we should access to daily returns for the aggregate...testing strats on the SP500 is a nightmare - trying to build an equity curve in excel is hugely time-consuming as you have to compile the daily returns for the 500 stocks....

Cheers

[NinjaTrader_Austin]

Hi there, I have forwarded your request to the right people.

[pmorissette]

Hi there, I have forwarded your request to the right people.

Thank you for the quick reply!

[NinjaTrader_Austin]

You are welcome. Enjoy the rest of your day!

[dave1992]

Can you please make it such that we can see the same stats (tabs) in the overall entry as the ones in the individual securities? The information seems to be there. At the very least, we should access to daily returns for the aggregate...testing strats on the SP500 is a nightmare - trying to build an equity curve in excel is hugely time-consuming as you have to compile the daily returns for the 500 stocks....

Cheers

I asked for this in this thread in February.

Since then I've had to implement my own solution to combine all the results and generate stats and graphs. It's a real shame that each user has to do this if they want to test multi instrument properly.

[toptrader]

Dave,
There is a product available at http://www.mooretechllc.com/Software...oAnalyzer.html that may be what you're looking for.

[dave1992]

Many thanks for that toptrader. Fortunately I don't need to backtest different strategies together, and for combined results I've already implemented what I need.

I'll wait for Ninja to put in what really should be offered by them rather than a third party partner. It's lucky I'm not cynical.... ;-)

[ChiTrader2000]

Combined Results: How to Have All Performance Tabs That Each Contract Has?

Hello,

Is this post's information still up-to-date... Is it true that there is no possible way to use NinjaTrader7 to view all the excellent performance tabs, such as "Graphs" and "Periods" with the very important charts and graphs, for the "COMBINED RESULTS" line item in a NinjaTrader back-test that was done over 1,000 futures contracts?

These tabs would be so much more useful if they were for the backtest of the entire basket of instruments, not just per-product. My backtests are over 1,000's of individual futures contracts, so the "Graphs" and "Periods" tabs are not very useful at a per-product level.

I really like the "COMBINED RESULTS" line item and it's "Summary" tab, but I am really hoping there is some way that either now or very soon this can be done in NT7.

Thanks in advance,

ChiTrader2000

[J_o_s]

Is it true that there is no possible way to use NinjaTrader7 to view all the excellent performance tabs, such as "Graphs" and "Periods" with the very important charts and graphs, for the "COMBINED RESULTS" line item in a NinjaTrader back-test that was done over 1,000 futures contracts?

As far as I know, this still isn't possible. I agree with you, this is a serious lacking feature of NinjaTrader and I'd also love to see it implemented in the future.

Perhaps other Ninja users already found an way around this?

Regards,

[NinjaTrader_Jason]

I am not aware of a workaround to view combined results in the various tabs of the Strategy Analyzer.

I will forward your feedback to Development as well.

[dave1992]

Jason,

Rather than giving it to development now, can you confirm that this has been with development for months now? I can tell you for certain, a significant number of professional traders would not be able to consider their backtest a success without looking at these combined stats. That is why I had to write my own code to combine the results and produce graphs, which is of course ridiculous, given that you probably already have the code to do this. I realise NT7 release takes priority, but in my opinion features should be implemented fully or left for a future date.

thanks
Dave