Inconsistencies in Strategy Analyzer

[pstrusi]

Hi Ninjas,

Strategy Analyzer really needs a deep revision. This week I've had a very annoying issues when doing several backtests. Basically the issue is, under the same conditions, just changing the end date, suddenly after few test, you find that some test with recent dates, has fewer winning or loosing trades than earlier dates backtest. So to eliminate this issue, I have to close NT, erasing the data cache and restart once again NT. As you must understand, this is very inconvenient when you're in active trading hours ( for live data downloading ) and because many backtests are proven to be wrong cause I didn't realize that detail in the number of trades, therefore I have to repeat many backtest which is a total waste of time.

I hope you take care of this as soon as you can

[NinjaTrader_Jason]

Hello pstrusi,

Sorry I am not quite sure if I understand. Are you performing backtests using the exact same strategy settings and exact same amount of data, which produce different results? If so, can please perform two identical simple backtests using the SampleMACrossOver strategy and check if results are the same.

[pstrusi]

Hi Jason, thanks

Yes, it is like that, the exact conditions, after many backtests these funny results begin to show, which is SOLVED when I close NT, erasing the database cache and restart NT again; then I can continue backtest with reliable results, till once again begin the flaw. If it was something of my script this would happen right away and it wouldn't be solved by doing a fresh start of NT.

Please just take the note to your developers, there's a flaw that needs to be addressed

[NinjaTrader_Jason]

After how many backtests does the issue approximately occur? I cannot reproduce it so far when I test the SampleMACrossOver. I will need to find a reproducible scenario before reporting to development.

The results in the Strategy Analyzer are produced by a combination of the strategy and historical data that is used. It seems historical data on your end changes after some time, since the issue is solved after deleting the cache.

Are you connected to a data feed when you perform the backtests?

Does it make a difference if you disconnect after the first backtest and perform subsequent backtests while disconnected?

[pstrusi]

Good suggestions, but I'll have to try that on weekends, now I'm dowloading live data that can't stopped cause it comes from IB.

Thanks