I've been running a simple trading strategy using the PFE indicator on ES 1 minute bars. The results of the real-time simulation and backtests are wildly different, but not just in price (i.e., not just a missed tick) - what I'm noticing is that the number of trades/day is also very different. Btw, I have read the page on difference in backtest/real-time simulation performance and this doesn't seem to explain it.
The fill logic is default and there are no limit prices being uses - hence, I believe I should be paying the spread and getting guaranteed fills in real-time simulation. However, even if the fill logic was faulty, I can't understand how the real-time simulation is adding trades that the backtest is not.
For instance, here is a comparison of 3 (bad!) trading days for the strategy:
1/3/11
Realtime: 17 trades, -$112.50
Backtest: 8 trades, -$200
1/4/11
Realtime: 10 trades, -$112.50
Backtest: 13 trades, -$125.00
1/5/11
Realtime: 14 trades, +$175.00
Backtest: 11 trades, +$612.50
Particularly in the case of 1/5/11, even if all 14 trades missed by 1 tick getting in and out (which seems a bit unlikely), that still doesn't explain the large difference in P&L or the extra "added" 3 trades.
I'm running NT 7.0 on a Dell laptop with 4 gigs RAM and my internet access is FIOS 25Mbps, so I don't think this is a slow hardware/connectivity problem.
Is it possible that some indicators such as PFE calculate differently during real-time versus historical backtesting and/or have bugs in them?
Other strategies also have this problem (won't go into all the details here) - but the backtest and real-time simulation seem to be running different signals and have very different entry/exit points. To make the problem more interesting, at least a few strategies with different indicators seem to conform pretty closely to the backtest - so it doesn't seem to be a problem that I'm generating consistently.
What further information can I give that would be helpful? Any ideas on how this is happening?
Is it possible that not calculating on bars close would somehow improve this issue?
Thanks
The fill logic is default and there are no limit prices being uses - hence, I believe I should be paying the spread and getting guaranteed fills in real-time simulation. However, even if the fill logic was faulty, I can't understand how the real-time simulation is adding trades that the backtest is not.
For instance, here is a comparison of 3 (bad!) trading days for the strategy:
1/3/11
Realtime: 17 trades, -$112.50
Backtest: 8 trades, -$200
1/4/11
Realtime: 10 trades, -$112.50
Backtest: 13 trades, -$125.00
1/5/11
Realtime: 14 trades, +$175.00
Backtest: 11 trades, +$612.50
Particularly in the case of 1/5/11, even if all 14 trades missed by 1 tick getting in and out (which seems a bit unlikely), that still doesn't explain the large difference in P&L or the extra "added" 3 trades.
I'm running NT 7.0 on a Dell laptop with 4 gigs RAM and my internet access is FIOS 25Mbps, so I don't think this is a slow hardware/connectivity problem.
Is it possible that some indicators such as PFE calculate differently during real-time versus historical backtesting and/or have bugs in them?
Other strategies also have this problem (won't go into all the details here) - but the backtest and real-time simulation seem to be running different signals and have very different entry/exit points. To make the problem more interesting, at least a few strategies with different indicators seem to conform pretty closely to the backtest - so it doesn't seem to be a problem that I'm generating consistently.
What further information can I give that would be helpful? Any ideas on how this is happening?
Is it possible that not calculating on bars close would somehow improve this issue?
Thanks
Comment