Same Strategy code gives different result in Strategy Analyzer than in Real-Time

[jzaba1968]

Hi,

I've written a piece of code to implement a strategy and now I would like to backtest it with the strategy analyzer. I did a backtest and found strange results in the strategy analyzer, so I went to run it in real-time using the Playback connection. Surprisingly I got a different result in real-time execution, which was more like the result I was expecting compared to the result in Strategy analyzer. I suppose it has to do with the way strategy analyzer implements the tick replay and builds the bars, but after searching the documentation and reading the "managed approach
2 and "advanced managed approach" I haven't found any solution yet. Can anyone help me with this issue? I'm attaching the results from both strategy analyzer and real-time executions. I've tried to upload also a short video showing how the relevant bar is built in real-time but it failed due to file size. In case it was needed I would provide the file through any other means. Thanks so much in advance.

[NinjaTrader_Gaby]

Hello jzaba1968,

Thank you for your post.

Live orders are filled on an exchange with a trading partner on an agreed upon price based on market dynamics. Backtest orders are not using these market dynamics. Instead these are filled based on logical rules from processing historical data.

• When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data.
  • This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known
  • Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Intra-bar granularity adds a second data series such as a 1 tick series using AddDataSeries() so that the strategy or indicator has the individual ticks in the historical data in between the High and Low of the primary series.

• This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with. The orders placed will need to use the BarsInProgress (BarsArray) index of that finer resolution series.
  • https://drive.google.com/file/d/12F6...C3TgPC3pA/view
  • Help guide link

In NinjaTrader 8, there have been two new enhancements so that programmers may not have to manually add this secondary series and code the script for high accuracy fills (Order Fill Resolution) or for intra-bar actions (TickReplay) depending on the needs of the script.
Note: bar types that are IsRemoveLastBarSupported cannot be used with TickReplay and generally cause inaccurate results when backtesting in historical data.
Note: High Order Fill Resolution allows for intra-bar order fills with historical processing, but is not compatible with Tick Replay.


Please see this forum post for a detailed explanation:

https://forum.ninjatrader.com/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live?p=773377#post773377

If you have any further questions, please let me know.

[jzaba1968]

Thanks for your update Gaby.

My strategy is coded using tick replay, so OnBarUpdate is supposed to be called on each tick. I have reported some pictures to show the difference between strategy analyzer and real-time, where it can be seen that in the strategy analyzer I get the StopLoss swipped out, whereas in real-time the price reaches the Profit Target. Having a look at how the bar is formed tick by tick, it's impossible that the price can reach the stoploss before the profit.That means that even if the strategy is coded as tick-based, and the strategy analyzer is set to execute on tick replay, looks like it still doesn't work properly in some cases.

In any case, I'll have a look at that post to see if I can find a solution there.

BR
Juan

[NinjaTrader_Gaby]

Hello,

Tick Replay does not provide fill prices for accurate order fills. Intrabar granularity still needs to be implemented so that the strategy has the individual ticks in the historical data in between the High and Low of the primary series.

Please let me know if you have any further questions.

[jzaba1968]

Hi Gaby, I have implemented intra-bar granularity by adding a second bars series (AddDataSeries(BarsPeriodType.Tick, 1). Unfortunately, the issue is still there. Let me share with you the files from both executions. From strategy analyzer execution you can see the trade info export to excel and also the chart picture. From real-time execution you can see the trade info export excel, the chart picture and also a short video with the bar building process (the bar where the entershortlimit order, the stop loss and the profit are submitted).

Real-time execution is working as I expected according to my idea and my code, however, strategy analyzer is working abnormally. You may notice that the stop loss filled in the strategy analyzer execution is in a price level where the market will not pass by until 3 bars after the entershortlimit order is filled. So it's physically impossible to reach the stop loss before the profit target which is 13 ticks below the entershortlimit order (still in the same bar).

Please see all files mentioned in the following link:

https://drive.google.com/drive/folders/1DqShNLmqEoioCc8fafpk7Ib-d-AJ0QNq?usp=sharing

Hope we can get to the bottom of the issue.

Thanks so much for your help
BR
Juan

[NinjaTrader_Gaby]

Hello,

If the expected trade(s) are not appearing, this would indicate that the condition to place the order is not evaluating as true and the order is not being submitted, or the order is being ignored for other reasons, or the order is being cancelled.

To understand why the script is behaving as it is, such as placing orders or not placing orders or drawing objects when expected, it is necessary to add prints to the script that print the values used for the logic of the script to understand how the script is evaluating.

In the strategy add prints (outside of any conditions) that print the date time of the bar and all values compared in every condition that places an order.

The prints should include the time of the bar and should print all values from all variables and all hard coded values in all conditions that must evaluate as true for this action to be triggered. It is very important to include a text label for each value and for each comparison operator in the print to understand what is being compared in the condition sets.

Prints will appear in the NinjaScript Output window (New > NinjaScript Output window).

Further, enable TraceOrders which will let us know if any orders are being ignored and not being submitted when the condition to place the orders is evaluating as true.

I am happy to assist you with analyzing the output from the output window.

Run or backtest the script and when the output from the output window appears save this by right-clicking the output window and selecting Save As... -> give the output file a name and save -> then attach the output text file to your reply.

Below is a link to a forum post that demonstrates using informative prints to understand behavior and includes a link to a video recorded using the Strategy Builder to add prints.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

Please let me know if I may further assist with analyzing the output or if you need any assistance creating a print or enabling TraceOrders.

[jzaba1968]

Hi Gaby, if you would be so kind to watch the short video that I provided to you before, you would see that the order is submitted. That's not the issue. Please have a look at it and let me know what you think.

Thanks
Juan

[NinjaTrader_Gaby]

Hello,

Unfortunately I am not able to watch the video because it says I don't have access.

Ultimately, in order to understand why the script is placing (or not placing orders) when expected, it will be necessary to debug using prints and TraceOrders. Without this, it's not possible to say with certainty why the script is executing the way it is.

Please follow the steps from my previous post to debug using prints and TraceOrders.

If you need any assistance creating a print, enabling TraceOrders, or analyzing the output please let me know.

[jzaba1968]

Hi, I'll follow your steps and get back to you later.

Thanks Gaby