Backtest data is different from live data

[tradingGary]

Hello NT, I am using NT Live 64bit 7.0.1000.6 version.

Issue:
I am using multi-instrument 1 minute bar chart with CalculateonBarClose = true. What I have found is that my Close prices for each instrument are different between production and backtesting env. My backtesting prices come from NT Historical Data Manager download using live zen-fire feed. My production prices are captured using the live zen fire feed.

The way I tested this is by writing an output file in the live and backtesting environments triggered in the OnBarUpdate method when BarsInProgress=0.

My CalculateOnBarClose = true; statement is the last statement in the initialize method.

My Goal:
Is to have the backtest use the same prices as it does in production to ensure same buy/sell signals are generated.

Appreciate your time in advance,
Thanks,
Gary

[NinjaTrader_Matthew]

Hello tradingGary,

Thank you for your post.

I appreciate the desire to have the same exact results in backtesting as well as in real-time, however this is just not possible. It is expected to see variying results from real-time to backtesting.

If you have not alreayd, I would suggest reviwing our Help Guide topic on 'Discrepancies: Real-Time vs Backtest':

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?discrepancies_real_time_vs_bac.htm

Please let me know if you have any questions.

[tradingGary]

Mathew,

Just to clarify, I realize that the fill mechanism is different, what I am referring to is the difference in Close prices between back test and live. I am trying to understand the cause of this difference (has nothing to do with generating trades)

Please let me know if you understand my question. I think the issue may be with the way the bars are formed on live and backtest but I am not really sure.
Thanks,
Gary

[NinjaTrader_Matthew]

Hello tradingGary,

Yes, I understand and you are correct that there are differences in the way the real-time data is processed and the way the the bars are stored on the historical data server.

To ensure the data you are receiving in real-time and the historical data is as close as possible, it is suggest to always ensure your local PC clock is in sync with the data server.

You can sync your PC clock by double clicking on the clock in the lower right corner of your desktop. Once you have done that, click on Internet Time tab and then click Update. Your PC clock should now be updated.

You can also keep an eye on any discrepancies in real-time by first opening a Time and Sales window and comparing the real-time time stamps to your local PC clock. You will want to compare this down to the second. Note: this would only be true if your data provider offers native time stamping. You can check on your particular data provider from the following link:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?historical_data.htm

I am also including a link to our Help Guide on 'How Chart Bars are Built' for more information:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?how_bars_are_built.htm

[tradingGary]

Mathew,
Thanks for the reply. I have ensured that my pc is synched with the internet. I have also verified that my zen-fire feed supports Native time-stamp.Given that the bars are time-stamped, should I then conclude that they should be matching between real-time and historical? Or does NT still uses local time to close the bar? Can you explain in more detail how this works.

thanks,
Gary

Hello tradingGary,

Yes, I understand and you are correct that there are differences in the way the real-time data is processed and the way the the bars are stored on the historical data server.

To ensure the data you are receiving in real-time and the historical data is as close as possible, it is suggest to always ensure your local PC clock is in sync with the data server.

You can sync your PC clock by double clicking on the clock in the lower right corner of your desktop. Once you have done that, click on Internet Time tab and then click Update. Your PC clock should now be updated.

You can also keep an eye on any discrepancies in real-time by first opening a Time and Sales window and comparing the real-time time stamps to your local PC clock. You will want to compare this down to the second. Note: this would only be true if your data provider offers native time stamping. You can check on your particular data provider from the following link:

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?historical_data.htm

I am also including a link to our Help Guide on 'How Chart Bars are Built' for more information:

http://www.ninjatrader.com/support/h..._are_built.htm

[NinjaTrader_Matthew]

Hello tradingGary,

You should expect your minute charts to match at this point. However any intraday intervals such as tick, range, volume, second, etc could still exhibit issues.

The charts will be built at the time the tick came in to NinjaTrader from your clock. If you were experiencing temporary latency in your internet connection, it would be possible for the incoming ticks to be off by a few seconds, which would then throw off the real-time results versus backtest.

There is no one of knowing exactly how the data provider stores the historical data you are downloading, so slight discrepancies in these cannot be avoided. However as discussed, you can relieve this by ensure you have the most accurate time on your PC.

[JamieHendrix]

Hi Matthew. I am having similar issues. My backtest results are really good and my Real time and market replay results are not only not as good but the chart in the strategy analyzer looks nothing like the same chart in the replay. I use 2500 tick charts. I understand the way charts are built and that there would be differences but they are not even close.

[NinjaTrader_RyanM1]

Hi JamieHendrix,

Unfortunately this is something you will see when comparing backtest / real time strategies. We expect that there are discrepencies in strategies real time compared to a backtest:

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

[JamieHendrix]

I understand Ryan thanks. Unfortunately the differences are so drastic that it renders all my backtesting invalid. Does the speed at which I run my Market replay have anything to do with the results, I usually run it at 30x to get more days tested. I have to get this straightened out because my algorithm is successful but I need to figure how successful (drawdown mostly) for position sizing.

Thanks for your help.

[NinjaTrader_RyanM1]

Backtesting is a good starting point for sure, and then real time sim or market replay should be the next step. If your backtesting results are drastically different than real time trading, then it could be there's some part of your strategy takes advantage of this lack of granularity or has been form fitted.

Generally you should expect the same results no matter the speed of market replay feed.

[JamieHendrix]

Thanks buddy. Using the analyzer for a starting point to see if it has validity is the way to use it. Real time is always different. Wish the analyzer could use more of a market replay type analysis.... that would be extremely helpful and more accurate..... food for thought.

Jamie Hendrix

[JamieHendrix]

Hey Ryan, I was wondering if there is any way to dump any saved data from NinjaTrader. Ever since I upgraded to NinjaTrader Standard (from the free version) it is very slow to load anything.

[NinjaTrader_Matthew]

Hello JamieHendrix,

Your historical data can be edited or deleted from the Tools--> Historical Data Manager

You can reduce the size of your database by going to Tools--> Options--> DB tab--> select "Reset DB". This will clear out any historical order, executions and ATM Strategy data.

You should also check that you are not running any unnecessary workspaces. When you go to File--> Workspaces everything listed below "Save As" is open and using CPU and Network resources. The workspace name with the check next to it is your active workspace and anything else listed here is in an inactive, but in an "open" state. You can delete any of your workspaces by going to File--> Workspace--> Open Workspace--> then select Delete on the workspace you wish to remove.

Click here to view the NinjaTrader performance tips

[JamieHendrix]

I have been running my automated system now in real time for over a month and everything has been working great. Thursday, I ran the system on my real money account and the simulated account to see if I could run two accounts at the same time. Things went wrong. there were orders that should not have been, I was even Long AND Short at the same time at one point.... things seem to go awry when I run two systems on the same account concurrently as well.... any reason for this? I have two versions of my system that I would like to run on each account that I have but I don't want these problems

[NinjaTrader_AdamP]

Jamiehendrix,

You could potentially be long and short if you have two accounts. Shorts will only close out longs and vice-verci if its on the same account. I would suggest really looking at your strategy logic and verifying orders step-by-step from the logic in your strategy. Are your accounts through the same broker? Who is your broker?

I would also suggest reading this help topic : http://www.ninjatrader.com/support/h..._account_p.htm

Please let me know if I can assist further.