Different Results between strategy analyzer and running strategy live

[samish18]

I am getting very different results between running a strategy live and the checking the results at the end of the day using strategy analyzer. No I am not talking about exit/entry price, I understand slippage, the live strategy is taking multiple trades a day when in strategy analyzer, no trades were taken

[NinjaTrader_Gaby]

Hello samish18,

Thank you for your inquiry.

To clarify, when using the Strategy Analyzer you are getting no results when backtesting your strategy correct?

If the strategy is in the Strategy Analyzer, is there data appearing on the Chart Display of the Strategy Analyzer?

Importantly, are there errors appearing on the Log tab of the Control Center?


Try testing the Sample MA Crossover strategy included with NinjaTrader on the same chart or Strategy Analyzer using the same instrument, bar type, interval, and date range.

Do you see results with the Sample MA Crossover strategy? (This would confirm for us data is available and the issue is with the logic in the custom script)


If the strategy is getting data and the strategy is enabled or a backtest is run with no errors in the Log tab of the Control Center, and the Sample MA Crossover is returning results, then would likely indicate the logic conditions in the custom strategy did not evaluate as true or orders are being ignored or cancelled.

In order to better understand how the code is working, it will be necessary to use Print to see how the conditions are evaluating and enable TraceOrders to see if orders are being submitted, ignored, rejected, or cancelled.

Below is a link to a forum post that demonstrates using prints to understand behavior and includes a link to a video recorded using the Strategy Builder to add prints.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

Enable TraceOrders, print the time of the bar and all values used in the conditions that submit entry orders. Include labels for all values and comparison operators.

Let me know if you need any assistance creating a print or enabling TraceOrders.

Save the output from the output window to a text file and provide this with your reply.

I'll be happy to assist with analyzing the output.

[samish18]

I am getting fewer results (sometimes 0) when backtesting on the same day I ran the simulation live. Data is appearing in the strategy analyzer and I am getting no errors in the log tab

[NinjaTrader_Gaby]

Hello samish18,

Thank you for your response.

A strategy running real-time (live brokerage account, live market simulation, Playback connection etc...) will produce different results than the performance results generated during a backtest.

Live orders are filled on an exchange with a trading partner on an agreed upon price based on market dynamics. Backtest orders are not using these market dynamics. Instead these are filled based on logical rules from processing historical data.

• When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data.
  • This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known
  • Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Intra-bar granularity adds a second data series such as a 1 tick series using AddDataSeries() so that the strategy or indicator has the individual ticks in the historical data in between the High and Low of the primary series.

• This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with. The orders placed will need to use the BarsInProgress (BarsArray) index of that finer resolution series.
  • https://drive.google.com/file/d/12F6...C3TgPC3pA/view
  • Help guide link

In NinjaTrader 8, there have been two new enhancements so that programmers may not have to manually add this secondary series and code the script for high accuracy fills (Order Fill Resolution) or for intra-bar actions (TickReplay) depending on the needs of the script.
Note: bar types that are IsRemoveLastBarSupported cannot be used with TickReplay and generally cause inaccurate results when backtesting in historical data.
Note: High Order Fill Resolution allows for intra-bar order fills with historical processing, but is not compatible with Tick Replay.


Please see this forum post for a detailed explanation:

https://forum.ninjatrader.com/forum/...377#post773377

Please let me know if you have any further questions.

[samish18]

Both strategies are price based on end of bar using all of the same settings. The issue is: when running a strategy live, it will take 6 trades in a day. When using that day as a sample for the strategy analyzer, it will take 2 trades.

[NinjaTrader_Gaby]

Hello samish18,

Thank you for your response.

In order to better understand how the code is working in the Strategy Analyzer, it will be necessary to use Print to see how the conditions are evaluating and enable TraceOrders to see if orders are being submitted, ignored, rejected, or cancelled.

Below is a link to a forum post that demonstrates using prints to understand behavior and includes a link to a video recorded using the Strategy Builder to add prints.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

Enable TraceOrders, print the time of the bar and all values used in the conditions that submit entry orders. Include labels for all values and comparison operators.

Let me know if you need any assistance creating a print or enabling TraceOrders.

Save the output from the output window to a text file and provide this with your reply.

I'll be happy to assist with analyzing the output.

[samish18]

This is the discrepancy I'm seeing between the two, as you can see, I am using market orders, but getting very different actions. All orders are filled

[NinjaTrader_Gaby]

Hello samish18,

I see you are using Renko bars. Are you using High Order Fill Resolution or implemented intrabargranularity?

With standard Renko bars, a backtest is very challenging because the bars have a function that removes the last bar and replaces it with a new Open price. This is difficult to simulate historically vs. how it occurs in real time.

You may get more realistic backtest results by using High Order Fill Resolution or by coding your strategy to submit orders on a single tick data series by adding intrabar granularity.

Order Fill Resolution - https://ninjatrader.com/support/helpGuides/nt8/index.html?understanding_historical_fill_.htm#Unde rstandingOrderFillResolution

Example script - Adding intrabar granularity - https://ninjatrader.com/support/help...ipt_strate.htm


Another option would be to backtest your strategy using the Playback Connection so that the bars may be built more realistically as if they were in real time.

Playback - https://ninjatrader.com/support/help...connection.htm


There is a section on the "Discrepancies: Real-Time vs Backtest" page in our help guide about backtesting Renko bars here:
https://ninjatrader.com/support/help...ime_vs_bac.htm

Please let us know if we may be of further assistance.

[madb123]

check my last comment here:

https://forum.ninjatrader.com/forum/ninjatrader-8/strategy-development/1293457-how-does-nt8-treat-enterlong-and-entershort-issued-on-same-bar-close

your issue is different being renko bars.. but if you missed that you probably dont know about these either.

[samish18]

Thank you! What exactly do you mean by Renko bars remove the last bar and replace it with a new open?

[NinjaTrader_Gaby]

Hello samish18,

Renko bars on realtime data re-draw the open based on the direction of the bar. This cannot be simulated historically, since backtests use only four data points: the open, high, low and close.

Please let me know if you have any further questions.

[samish18]

I see what you mean about redrawing the open. If a strategy is executing on bar close, why would this impact entries?

[NinjaTrader_Gaby]

Hello samish18,

Renko bars are built the same way regardless of the strategy's calculation mode. In historical, the same data isn't available that is available in realtime which affects how the renko bars will be built.

This is why regardless of your strategy's calculation mode, if you are backtesting using Renko, we recommend either using High Order Fill Resolution or implementing intrabar granularity. Or using Market Replay data so the bars will be built more realistically.

It is more a matter of the difference in data in historical vs. realtime rather than Calculation mode.

Please let me know if you have any further questions.