Asking for members to comment (not NT): How close is sim test on live market

[Austiner87]

Hi Everyone, (this question was answered by NinjaTrader in another thread, but they (understandably) cannot answer with more specificity. I very much appreciate the answer NinjaTrader gave, but looking for more responses from members. thank you!

I have been testing a strategy on the live market but with a sim account. Can anyone comment on how accurate this is when compared to trading with real money/real contracts on the live market? For instance, if I ran a strategy on a sim account and the same strategy on a real-money account on an actual trading day in real-time (no backtest) how close should I expect their end-of-day PnL's to be?

Is it within, say, 90% 95% or 99% accuracy? OK if lower, just trying to assess the accuracy rate. Assume there are no problems with margin. Thank you!

[Austiner87]

Bumping this to the top, would appreciate any comment, thank you!

[jeronymite]

Austiner87 They will be different, that much is certain. Let's assume everything mentioned in the following:

• https://ninjatrader.com/support/help...simulation.htm
• https://ninjatrader.com/support/help...ime_vs_bac.htm
• https://ninjatrader.com/blog/break-v...ive-trading-3/
• https://ninjatrader.com/blog/why-do-...d-stop-orders/
• https://ninjatrader.com/blog/hide-or...ed-orders-mit/

Some general considerations:

• Simulation by nature has random timing for various things, most notably movement through Order States and time-to-fill
• Timing for Live Orders is heavily dependent on infrastructure that Sim does not experience: network, servers, routing, exchange, order book queues, etc
• If the market data source in Sim is different from the actual Live data from the broker (quite likely), prices will almost certainly be different and so results will be also
• Slippage in Sim may not align with the Live market
• Sim has a variety of fill algorithms, so even within Sim alone there may be differences
• Volume will probably vary: Sim may "invent", Live does not
• Sim may allow various Order types that your broker may not support
• Using synthetic (or simulated) Orders in the Live market (e.g. MIT or simulated Stops) may behave differently from in Sim
• The psychology of trading Sim vs Live is also important to take into account

Hope that's useful input for you.

Thanks.