Backtesting and live (paper) trades are different

[lookOutBelow]

After testing against 10/19's live/paper trading and backtesting the same date, I can see that the trades are off quite a bit again. I also downloaded Market Replay Data for 10/19 and had a third set of different results!

The link provided by Jason gives some examples of why tick data can be different from backtests and live and even how two backtests can be different. My question is, is it more likely to be closer to accurate if I use minute bars instead of ticks? Or do we have the same issues either way?

[dave1992]

Yes, happens to me too. Actually I couldn't develop properly without that print statement, and all my strats have one. It's a bit of a hassle, but invaluable for going from backtest to replay to live. There are plenty of diffs between replay and live too! On the plus side, when you sort out these diffs you occasionally spot something important in your logic that you'd missed.

[lookOutBelow]

If trades enter at wildly different times isn't it a fairly simple task to have a print statement outputting all the criteria and just compare them?

yeah. If after today, I see the same weird problems, I'll have to do that. There has got to be a reason for this. Maybe, printing out every single value at the time of the trade would help. I'm just overly frustrated. This isn't the first time I've had issues with backtesting giving me strange results from one minute to the next.

[dave1992]

If trades enter at wildly different times isn't it a fairly simple task to have a print statement outputting all the criteria and just compare them?

[lookOutBelow]

No, there is something else going on. The trades are triggering at completely different times under completely different circumstances. Like I said, it isn't a slippage thing. And not a volume thing either. I understand the differences between live and backtesting and how fills are made and bars are built. This seems to be something else.

I'm using market orders, so it should be filling immediately, even if the price is way off. So, that isn't the case either when you see 1 hour difference in an extremely liquid (ES for example) instrument.


I'm going to let it run again today and more closely scrutinize the differences. I've had problems in the past with backtesting giving me different results after no changes to parameters. Something is still screwy to that effect and may be related.

[NinjaTrader_Jason]

Hello lookOutBelow,

Please see the link below for more information why you can expect differences between backtests and running strategies live.

http://www.ninjatrader.com/support/helpGuides/nt7/discrepancies_real_time_vs_bac.htm

[dave1992]

What type of orders do you use? Have you compared against a market replay, which should be closer to live than backtest. Live and replay have bid ask volume, which can make a difference - check out the Ninja page on the difference between live and test.

Don't forget that if your live fails on one trade because there's no volume at that point, then your setup is no longer valid then you might take a trade 10 mins later that your test didn't because you weren't flat in test. A whole set of trades can differ because of that.

[lookOutBelow]

I'm having a weird issue and I'm not quite sure what to make of it.

I'm seeing differences between live trades and backtests. The discrepancies are much more than slippage. Both are using 150 tick charts and the same session, same strategy, and same instrument. However, the entries/exits times and prices are quite a bit different.

This happened on all 7 of my strategies that ran throughout the day today on 7 different instruments. Only 1 of the strats was in the same ballpark with trade times and entry and exit prices.

I've attached an example image of what the backtest said and what the account performance said from the live trading after the day was over.

Thanks