Backtesting results v7 vs v6.5

[NinjaTrader_Bertrand]

Could you repost the 6.5 results please, I only see the 7 ones...also : are you using exactly the same base data set / data provider in both versions? Have you tried comparing the default SampleMACrossOver from us? Would that be closer together on your setup?

Thanks

[lookOutBelow]

Could you repost the 6.5 results please, I only see the 7 ones...also : are you using exactly the same base data set / data provider in both versions? Have you tried comparing the default SampleMACrossOver from us? Would that be closer together on your setup?

Thanks

Attached is the 6.5 results. And yes, this is using the same data set and provider. These results are from the SampleMACrossOver that is provided with NT.

[NinjaTrader_Jason]

I tested it on my end and results were much closer than yours, see attached screenshots.
The difference should not be as big as the results you display.

Can you please forward me a screenshot of the strategy settings you used as per backtest screenshots at post #15 and #17. Subsequently export the historical data from the Historical Data Manager that was used by the backtest in NinjaTrader 7.

Please send the screenshots and historical data to support[at]ninjatrader[dot]com. Reference this forum thread and put 'Attn Jason' in the subject header.

[lookOutBelow]

I tested it on my end and results were much closer than yours, see attached screenshots.
The difference should not be as big as the results you display.

Can you please forward me a screenshot of the strategy settings you used as per backtest screenshots at post #15 and #17. Subsequently export the historical data from the Historical Data Manager that was used by the backtest in NinjaTrader 7.

Please send the screenshots and historical data to support[at]ninjatrader[dot]com. Reference this forum thread and put 'Attn Jason' in the subject header.

I will do that as soon as I get a chance.

Out of curiosity. Even though some of the results are similar on your two runs. Is the difference in $$ acceptable? While your results are no where near as radically different as mine, even your results seem like a pretty big difference. Even in your example, you would of lost twice as much if you had used 7.0 instead of 6.5. This alarming to me.

This doesn't make me very confident regarding NT. The two versions are looking at the same data quite differently..??!! I'm nervous that when I'm forced to move to 7.0 I'll have strategies that are less effective. And even if I re-optimize and change things, I'll always be wondering in the back of my head if I'd be making more w/ 6.5.

[NinjaTrader_Jason]

Please note that there will be differences between backtesting strategies and running them live, see the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

The backtest in NinjaTrader 7 contains 27 trades more which causes the difference.

[lookOutBelow]

Please note that there will be differences between backtesting strategies and running them live, see the link below.

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

The backtest in NinjaTrader 7 contains 27 trades more which causes the difference.

I understand. But shouldn't the differences between 7.0 and 6.5 be minor? If they are both backtesting using the same data? Does 7.0 look at data that much differently than 6.5? I'm totally ignoring live or market replay. Speaking only of back testing on both versions.

[dnoff]

I was interested to understand from the NT team, the answer to the question above on why there would be differences between 6.5 and 7.0 when backtesting the same strategy/data??

I guess are there changes made that make it more/ess accurate, if what are they so we can understand the impact on testing scenarios etc???

[NinjaTrader_Jason]

Hello dnoff,

If you backtest the same strategy using the exact same settings and historical data, you should not experience differences.

Please note that NinjaTrader 7 uses session templates, while NinjaTrader 6.5 does not. Make sure that the session template selected in NT7 uses the same times as you would select during a backtest in NT6.5.

[lookOutBelow]

I was interested to understand from the NT team, the answer to the question above on why there would be differences between 6.5 and 7.0 when backtesting the same strategy/data??

I guess are there changes made that make it more/ess accurate, if what are they so we can understand the impact on testing scenarios etc???

This issue was taken care of via email exchanges. It seems my problem was that some historical data was screwed up. Either it wasn't properly transferring from 6.5 to 7, or something wasn't getting deleted all the way. Or some other vodoo that I'm still not clear on.

Eventually the problem was rectified and I got pretty similar results between the version (only off by a couple hundred $$ and a small number of trades).

Side note:
I'm having a similar problem between 7.0 beta versions .20 to .21. I still don't quite understand why the historical data is getting messed up. But the same strategy is getting huge differences in results from the same parameters/sessions/etc.

[alexgg]

hi

i noticed quite a big change between nt7.0 beta versions .20 to .21

this with same data as before new version

it seems to be firing off multiple orders some days

im not sure if this because it is stacking unique entries when it shouldn't ?

does cancel orders functionality work perfectly ?

is there any way to roll back to beta .20 ?

rgds

alex