Backtest vs. real-time demo results

[Borsz]

Hi,

Results of backtest and real-time demo running differ significantly, in case of any of my own-developed Ninja scripts.
Maybe something is not set properly for realtime demo running.
What settings should I check and adjust to ensure my robots enter and exit exactly as it did during backtest?

Thank you.

[NinjaTrader_Bertrand]

Borsz, some differences would generally be expected. We have collected common items that contribute on this link - http://www.ninjatrader.com/support/h...ime_vs_bac.htm

[Borsz]

Thanks Bertrand, I know these general factors which affect realtime result.
I meant exact settings (e.g. in NT/Tools/Options) which must be properly set before running a strategy in realtime sim or live. Do you have a this kind of "TO DO" list of pre-set parameters before go realtime?

My results are so opposite in realtime simulation and backtest (after plenty of trades executed), that the reason cannot be some "accidently not filled" trades.

Thanks.

[NinjaTrader_Bertrand]

If you trade live then there would be nothing else to set, the fills will come back to your strategy directly as result of realtime market dynamics. On the simulator though you can set a few options under the Tools for example how delays and partil / immediate fills should be factored in.

One thing in addition you always want to be critically aware of is the sync between strategy and account position(s) - http://www.ninjatrader.com/support/h..._positions.htm

For which market / timeframe do you currently test live? Which order types do you use?

[Borsz]

Thanks for infos, Bertrand.

I ckecked the link and know these possibilities when going live.

Although, in Tools/Options there are lot of settings which create questions in my mind. Does NT has detailed on-by-one description of each settings of Options menu?

I do live test on ES 06-14, 2- or 4-Range Bar.
Fill type: Default, Slippage: 0

[NinjaTrader_Bertrand]

Borsz, a documenation on the Tools > Options submenu content would be found here - http://www.ninjatrader.com/support/h...t7/options.htm

With a live test you refer to running in live simulation or executing to a connected live brokerage account? Slippage in your backtesting is 0, so if you're using market orders on Last price data in backtesting that would not be accurate.

[Borsz]

Yes, I use market orders in the script (simply "Enter long" and "Enter Short"), but where is that adjustable "Last price data" you mentioned?

Thanks for infos.

[NinjaTrader_Bertrand]

Borsz, I was referring to the chart data that you use, per default that would be based on the last price (option when you create the chart in the DataSeries window 'Price Based On'). So in backtesting that's what the historical fill processing algorithm could work on and why we have a Slippage field to rectify not working directly with bid / ask data (which is possible as well but only through additional custom coding). In your realtime trading / simulation the fills will come back as function of the Level 1 bid / ask seen live, so with a market you will be buying the ask and selling the bid.

[Borsz]

Bertrand, I GOT THE PROBLEM!

So, I compared trades drawn on two charts, for the same period:
1. chart with backtest trades
2. chart with trades executed and drawn during realtime demo simulation running.

All tardes drawn on these two charts are totally the same. Same entries, same exits.
But, while results of backtest trades were sumed up correctly (and strategy made profit), results of realtime running trades were sumed up on a totally wrong way. Complete error. Trades on the chart and trades in the Account performance/Trades tab have absolutely no any relation. Full difference. I don't know what trades are in the Account performance/Trades tab, but it seems it is totally bullsh!t.

Do you have any idea, why?

Thanks for your help.

[NinjaTrader_Bertrand]

Borsz, thanks for the follow up. The account performance would not filter / tag the execution together on a strategy basis (like the strategy performance would do), it would be basically doing FIFO (First in first out) processing, thus manual positions could contribute (and would make it harder to exactly follow the strategy). I would test on a dedicated sim aco**** for example where you only execute this strategy to, you can create yourself sim accounts under the Tools > Options > Simulator tab.

http://www.ninjatrader.com/support/helpGuides/nt7/multiple_simulation_accounts.htm

Then only execute to this account via your strategy and take extra care to ensure the sync is correct at alles times.

http://www.ninjatrader.com/support/helpGuides/nt7/syncing_account_positions.htm

This is the second item that could have caused an issue in your present setup tracking the strategy realtime actions - a non concern in any backtesting.

[Borsz]

No no no, I dedicated this Sim account only only for this strategy.
I even opened a new demo account to run my strategy in real-time.
No manual trades, no other strategies made any other trades.

[NinjaTrader_JC]

Hello Borsz,

Inside of your Strategy are you reversing your position without setting any exit orders?

Also, do you mind posted a screenshot of the two different results so that we may view it?

[Borsz]

Yes, there are two exit rules: SL or TP which rarely reached, and exit if run into opposite sign (reversing position), that happens often.
Why? How can it affect the account performance calculation?

Thank you.

[NinjaTrader_JC]

Hello Borsz,

This may be due to when Backtesting or on any Historical data processing on bar types like LineBreak, Renko, Range etc can return results that would not reflect what would happen in the real-time market (tick by tick). When running on Historical Data you just get a snap shot of the bar that printed not each tick that made up that bar. So for example; you have an 8 tick stop loss set, and you run a backtest on a 10 range bar type. Even if the market had moved down to your stop loss it would not fill because you are viewing only the bar (the 10 range in this case).

The most accurate results for testing your strategy is by using the Market Replay feature inside of NinjaTrader. The link below goes over the Market Replay that you may view.

http://www.ninjatrader.com/support/helpGuides/nt7/market_replay.htm

You may want to backtest your strategy with intrabar granularity for better fills, and an example of this can be found at the following thread.

http://www.ninjatrader.com/support/forum/showthread.php?t=6652

Can you compare your real-time results with Market Replay results and get them to match?

[Borsz]

Thanks JC.
I am going to learn the links and do market replay backtesting.

Anyway, both during backtest and realtime simulatin executions happened/happens on bar close.
The backtested and realtime tested frame is 2-range bar, so very small movements are inside bars, additionally SL/TP is reached about once of 50 trade. Almost every exit happens because of a new opposite entry signal.

I just went through one-by-one on the trades and e.g. in the last 1.5 days all trades matches (around 30-40 trades), but once treades in Account Performance / Trades tab suddently start to fully differ from trades drawn on chart. Interesting.