I am curious about a seasoned NT tech (say over 7 years experience) with strategies?

[birdog]

Hi folks,

I am curious about a seasoned NT tech (say over 7 years experience) with strategies & their opinions or observations of the following:

We all know we can have backtests that show good results backtesting but not so much realtime sim or live trading.

My question is, have you seen backtests that show terrible backtesting results but very much successful realtime sim and live trading?

Any details, insights, comments, etc you could share would be appreciated.



Greg

[noshowstock]

I am new to this (couple years in the making of a strategy) and I have seen back test results being sub par but real world results are better. But my strategy requires a relatively short term and the back test can only get minute data - so for example you might have a successful trade in one minute bars but the back test doesn't know how the market actually moved within that 60 second bar. Longer time frames and proper volume to be able to handle your orders and I think the back test can be pretty accurate. Do not expect to build a scalper algo and have realistic back test results.

[birdog]

NinjaTrader_BrandonH could you flag a NT tech that has over 7 years or so experience on this one? Also, I would love your opinion as well as I value your input highly. And, FYI, this is question is definitely not intended for scalping.

[NinjaTrader_BrandonH]

Hello birdog,

Thanks for your post.

The results that are seen when backtesting custom strategies in the Strategy Analyzer may be better or worse than the results seen when running the strategy on live data.

In the Support department, we would not be able to say for certain how a strategy will behave when comparing its backtest results to live results. This would depend on multiple factors such as how the strategy places and handles orders, the incoming market data, the current market conditions, and more will affect the results of a strategy.

The Strategy Analyzer would be used to backtest a strategy on historical data to see how the strategy might have performed on that past data. The strategy results would be different when running it on live data since this new live data is different than the previous historical data.

Please review the help guide document on the differences on real-time vs backtest (historical).
http://ninjatrader.com/support/helpG...ime_vs_bac.htm

Note that when using the Strategy Analyzer only the Open, High, Low, and Close will be available and there will be no intra-bar data. This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known.

Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

To improve the accuracy of a backtest, you may use Tick Replay along with an added 1-tick series to have logic processed intra-bar and have orders filled intrabar. Tick Replay would be used to have the logic process OnEachTick or OnPriceChange with historical data, but this does not allow for intra-bar order fills. You would need to add a single tick data series and submit orders to that single tick data series for a strategy that uses Tick Replay.

Please reference the SampleIntrabarBacktest example and the following Help Guide links for more information.

SampleIntrabarBacktest 'Backtesting NinjaScript Strategies with an intrabar granularity' - https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

TickReplay — https://ninjatrader.com/support/help...ick_replay.htm

Developing for Tick Replay -
https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub= developing+for+tick+replay

Please let me know if I may assist further.

[birdog]

NinjaTrader_BrandonH Got you. I know some of this, but just wondered what your experience has been over the years on it.

Regardless, what about us using 1-second bars but then using like a 1 min bar cross of the second bars to be able o even more reasonably get more precise entries too even with using Strategy Analyzer that way? I have also been testing that.

I know we can download tick data too etc for market replay.

[NinjaTrader_ChelseaB]

Hello Greg,

With 1-tick intra-bar granularity implemented properly, the results of each trade are typically within 1 tick, as long as in real-time there isn't a lot of latency.
There can be slight differences, and these can add up, but as far as finding the behavior of the strategy, backtesting is still useful for understanding how the strategy will react to data. The precise fills may be slightly different, but the strategy behavior is what you are trying to understand.

The information Brandon is providing you is correct.

I've made a video on it.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance#post100192

[waltFX]

Just wanted to mention that Forex back testing is somewhat more inaccurate since "bid" and "ask" are used for Long & Short trades (according to the docs) so you MUST insure that your history includes this data AND that these values are used for your trades...

>> This will definitely change results when using real-time data.

Truthfully, I am using "Close" in back-testing as my provider does not have bid/ask in their history so all strategies are tested in real-time with a paper account.

Walt