Market Replay vs Back Test results

[Malletto]

I just sent an email. Let me know if you received

[NinjaTrader_BrandonH]

Hello Maletto,

Thanks for your notes.

You could reply to the email ticket that we had opened to send the image to me.

Further, please test the strategy I shared using the same steps and settings seen in the demonstration video and confirm if the strategy is behaving the same way seen in the demonstration video.

[Malletto]

It won’t work. Can we email so i can send it

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

You could share the image or demonstration video on this forum thread.

To clarify, have you tested the strategy script I shared in post # 86 using the same steps seen in the demonstration video I shared?

Do you see a difference when testing that test script I shared using the steps I provided?

If the test strategy I shared is behaving the same way seen in the demonstration video I shared, this means it is likely the logic of your script causing discrepancies. Without debugging the logic of your script to find the line(s) of code causing the differences, there is nothing specific I could advise on.

It would be up to you to debug the logic of your script to understand exactly how it is evaluating your custom logic. It is against our policy to debug a script for you on your behalf in the Support department at NinjaTrader.

If you need someone debug the logic of your strategy on your behalf, you can also contact a professional NinjaScript Consultant who would be eager to assist you with your script.

You can search our extensive library of NinjaScript consultants through the link below. Simply enter a consultant name or search by using our filter categories. Once you have identified your consultants of choice, please visit each consultant's site for more information or contact them directly to learn more!

https://ninjatraderecosystem.com/sea...mming-services

Educators - https://ninjatraderecosystem.com/sea...ures=education

You can locate the contact information for the consultants on their direct websites for any additional questions you may have. Since these consultants are third-party services for NinjaTrader, all pricing and support information will need to be obtained through the consultant.

This NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The companies and services listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.

[Malletto]

How can i send it to you then. I want to show you what im seeing

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

I have not received an email from you that I can see.

If you are seeing differences then you must debug your script by adding prints to determine what exactly is evaluating differently. This would be up to you to do.

Note we do not offer debugging services in our support.

Read through the forum thread about comparing historcal, realtime, and playback results, add prints to the script that prints out ALL of the logic being used for calculations in the logic of the script, and test your strategy to see where differences might be. The forum thread below details how to accomplish this is contain a sample script you could view.

https://forum.ninjatrader.com/forum/...nce#post100192

Review this forum thread below demonstrating how to use prints to understand the behavior of a script's logic.
https://ninjatrader.com/support/foru...121#post791121

[Malletto]

Thank you for all of this. Did you see my email? I sent you the difference

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

The Strategy Analyzer can only test historical data, so this is what you would be comparing the backtest to on the live chart.

I let the strategy run for a little while on the live chart window. Then, I ran a backtest on the strategy on the Strategy Analyzer and see the dots drawn on the chart were the same as historically as they were on the live chart.

Further, after running another backtest the Trend Magic plot of the Live chart at 11:54 in the Mini Data Box window matches the Trend Magic plot of the Chart display of the Strategy Analyzer at 11:54. The bar values also match.

Demo video: https://brandonh-ninjatrader.tinytak...NV8yMjQ5NDI4Nw

Ultimately, if you see your logic is calculating differently then it would be up to you to debug your strategy's logic to see what the differences are in your custom logic of the strategy. This is detailed in the forum thread below which also contains a reference sample.

https://forum.ninjatrader.com/forum/...nce#post100192

Below is a link to a forum post that demonstrates how to use prints to understand behavior.
https://ninjatrader.com/support/foru...121#post791121

[Malletto]

Yeah i watched the video, for the sake of this experiment, can you let it run live until another change happens. What your showing is only historic movement

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

As previously stated, I added the strategy to a live chart window while connected to realtime data. Then, I ran a backtest on the strategy in the Strategy Analyzer using the same strategy settings and data series settings as used on the live chart.

Then I compared the results and see the bars, indicator plots, and dots being drawn by the strategy match. Showing the strategy is working the same historically as is it real-time.

See the demonstration video which shows the strategy run on the Strategy Analyzer.

Note that a backtest can only test on past historical data, not incoming live data. You could take the same steps seen in my demonstration video to test the sample script attached in post # 86 and to test your script.

You must add debugging prints to your script to understand exactly how your logic is behaving historically compared to realtime.

This forum thread talks about comparing historical to realtime to playback results and adding prints to a script to find where differences might be, if any. Review this forum thread and sample script, and add debugging prints to your strategy to understand how logic is evaluating.

https://forum.ninjatrader.com/forum/...nce#post100192

Below is a link to a forum post that demonstrates how to use prints to understand behavior.
https://ninjatrader.com/support/foru...121#post791121

[Malletto]

Could you also run the results on the strategy analyzer and let me know what you get

[Malletto]

I appreciate this a lot. Are you comparing with real-time, or is that just loaded on the chart. Because i notice when it’s loaded on the chart it’s identical to the backtest. But when it’s real-time, that’s when you see the difference.

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

Are you referring to setting the Calculate mode in the UI of the Indicators window?

Your strategy will use whichever Calculate mode the indicator is set to use in the code of the script itself. You would see what Calculate mode an indicator uses by noting the 'Calculate' property when adding the indicator to a chart.

I have imported the Trend Magic indicator from the Ecosystem User App Share linked below and created a simple test strategy using this indicator. The way the test strategy works is when the Close price crosses above the Trend Magic plot, a blue dot is drawn 2 ticks above the High of the bar.

Trend Magic: https://ninjatraderecosystem.com/use...d/trend-magic/

I have tested this sample strategy by running it on a chart while connected to a live data feed connection. Then, I ran a backtest on the strategy using the exact same settings as the real-time strategy and data series.

When testing this script, I see that the bars on the Strategy Analyzer Chart display match the bars on the live chart. The indicator plots on the Chart display of the Strategy Analyzer matches the live chart's historical data. The dots drawn by the strategy when the condition becomes true also match.

See this demonstration video: https://brandonh-ninjatrader.tinytak...Ml8yMjQ5NDIyOQ

Attached is the simple sample script used to test this. You could try testing the attached script using the same settings and steps seen in the demonstration video.

If you are seeing different behavior occur when running your custom script then it would be up to you to debug your script's logic to see exactly how your custom logic is evaluating between realtime and backtests to see where differences might be.

If your script was programmed by a third-party developer, reach out to that developer directly so they can debug the script they created.

[Malletto]

But this calculation is different from when you just turn it on through data series, correct?

[NinjaTrader_BrandonH]

Hello Malletto,

Thanks for your notes.

The Calculate mode is set within the code of a NinjaScript.

The difference is due to how the script's logic is processed between backtest and real-time.

Enable Tick Replay when backtesting so the indicator calculates tick-by-tick on historical data as it would when running real-time.