Historical data issue. Different backtest results between instances.

[martyn73]

I have NinjaTrader installed on two computers. When backtesting I run each backtest on at both computer because sometimes I get vastly different results between instances. It seems that historical data can somehow easily get corrupted or perhaps is missing data. I would like to understand how/when this can occur?

When I notice a significant difference between an identical backtest run on two different instances of NT I go into the NT /db folder and clear out all data from the cache folder and delete all historical data from their respective folders. Once I do that and re-run the backtest I get the same results between instances. I'm wondering if there is a better solution to avoid getting back incorrect backtest results?

[martyn73]

Also want to clarify that I only run backtests using time bars (1, 15, 30 minutes etc) not calculated bars like renko.

[QuantKey_Bruce]

Yes, in some cases, it is not clear to the platform that data is missing so it does not request the fill-in data and one side or the other is missing a section. Additionally, though, this is a lesser issue, there are correction ticks at the exchange which can mean the data captured in realtime differs from the data downloaded from the historical servers even if the historical servers used the same realtime data. Additionally, if one of them has a different realtime data feed from the other (for instance, Rithmic vs. Continuum) I would expect the data captured in realtime to be pretty different. Additionally, if one is capturing in realtime while the other only got the data historically, I would also expect that to be different. These are just some of the reasons - there are multiple reasons the data could and would be expected to be different - especially if any of it was captured in realtime.

One approach you could take to this is if you are only back-testing on computers that get their historical data from the same source, do your back-testing after hours, and do a reload historical data on both sides before you begin.

[NinjaTrader_BrandonH]

Hello martyn73,

Thanks for your post.

QuantKey_Bruce is correct. In some cases, there could be data missing between your machines and the platform does not detect if there is missing data.

We are tracking interest in a feature request for the ability to autodetect and backfill missing historical data in NinjaTrader 8 and I have added your vote to this feature request. This request is being tracked under the number SFT-4225.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted in the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

You may try redownloading the historical data on each machine prior to running your backtests as you have explained. Or, you could use QuantKey_Bruce's workaround for this.

Please let us know if we may be of further assistance to you.

[martyn73]

Thank you both for the reply and adding me to the feature request.