historical data best practices

[trader3000a]

Hi,
I find that after I run backtests, optimizations, wf-optimizations many times, that the results are no longer consistent. Should I assume this is due to corrupted historical data, and that due to data being overwritten to often? What are my best practices for keeping the data clean, purging the db, etc. ?
thanks,
David

[timmbbo]

First, I would reboot your machine between large runs. NT uses the .Net framework, which leaks memory like a sieve. Second, remember that there is a degree of randomness if an order is filled, or not.

[NinjaTrader_BrandonH]

Hello trader3000a,

Thanks for your post.

As a general rule for backtesting: Same Input data + Same strategy code and parameters = Same results.

Something you could do to ensure you have no missing data would be to re-download Historical data for the dates you want to test your strategy on (Tools > Historical Data > Load tab). If you notice that you are missing data for a certain date(s), the data will need to be re-downloaded for that date(s).

I've included some additional resources on the differences between historical and realtime data and further reading on these discrepancies that should be minded while backtesting. My colleague Chelsea's post goes into depth on creating strategies and backtests that produce consistent results.

Discrepancies between backtesting and realtime data — https://ninjatrader.com/support/help...ime_vs_bac.htm

Comparing real-time, historical, and replay performance — https://ninjatrader.com/support/foru...d.php?t=102504

See this forum thread for information about Walk-Forward Optimization results: https://forum.ninjatrader.com/forum/...59#post1160759

When running Optimizations there are a number of factors that are involved and memory utilization can climb very quickly depending on a few of these factors. Once memory gets maxed out, we can experience short freezes where memory is decommitted, stored to disk, and then new resources are committed before the backtest is resumed.

We can easily control the number of iterations involved and we can also consider writing our strategies to use IsInstantiatedOnEachOptimizationIteration = false; (Which requires that we reset class level variables in State.DataLoaded.)

IsInstantiatedOnEachOptimizationIteration — https://ninjatrader.com/support/help...niteration.htm
Optimization Tips — https://ninjatrader.com/support/help...ionPerformance
Walk Forward Optimization — https://ninjatrader.com/support/help...ss_metrics.htm
Genetic Optimization — https://ninjatrader.com/support/help..._algorithm.htm

[trader3000a]

thanks Brandon

[tkaboris]

Hi I also want to make sure, does downloading historical data with tick data equals to a market replay data quality?
I can only go couple of months in market replay so if i want to download longer period i should download historical with tickdata and that should equal to market replay?
Thank you

[NinjaTrader_BrandonH]

Hello tkaboris,

Thanks for your notes.

No, Historical Tick data would not be equal to Market Replay data. Market Replay data is the most accurate and holds both level I and level II (market depth) data.

See this help guide page for more information about Market Replay data vs Historical Tick data: https://ninjatrader.com/support/help...ePlaybackWorks

[tkaboris]

ok thank you for clarifying.
So is there a way to get market replay data for extended time like 3-6 months? or it has to be purchased somewhere.. What do you recommend?

[NinjaTrader_BrandonH]

Hello tkaboris,

Thanks for your note.

Only up to 90 days of Market Replay data is available to download from the NinjaTrader servers.

If you would like to use data further back, you would need to use Historical Tick data.