Playback vs Strategy Analyzer. Any limitations?

[Berry Trader]

I wrote a tick timeframe based strategy directly in the Ninjascript editor that runs without errors in real time and also without errors during playback. When I choose this strategy in the strategy analyzer and choose the same instrument and time frame, it returns zero trades over the same period that worked well in playback and in real time. There are no error messages and nothing in the log file. I have downloaded playback files for these days, but I think that the strategy analyzer runs against historical data I have not downloaded historical tick data, but when I check the historical files, it looks like they are there (see attached image). A full compliment of tick data is also there when I look at documents/ninjatrader 8/db/tick

Main clue: none of the scripts I have shows any result in the strategy analyzer over any time frame

I am running 8.1.3.1 64-bit. I have a Rithmic connection for my data (Apex Eval Entitlement). I have right clicked on the chart where my script runs well in real time and selected "Download all historical data"

Q1: Is there a simple demo strategy that I can download that will help me test if the strategy analyzer has the proper tick based data to run and that I know how to set the parameters in strategy analyzer? I want to see a known good script file running in the strategy analyzer before I assume there is anything wrong with scripts I wrote in Ninjascript editor.

Q1o you have any other suggestions about how to diagnose why the strategy analyzer seems to do nothing.

Thanks.

[Berry Trader]

Here's the image I mentioned.

[NinjaTrader_Gaby]

Hello BerryTrader,

Thank you for your post.

To get a better understanding of why nothing happens when you test in the Strategy Analyzer, please answer all of the following questions:

• What version of NinjaTrader are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.?.?.?)
• Do you see similar results when running the same test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?
• Who are you connected to? This is displayed in green in the lower-left corner of the Control Center window.
• Are you connected to your data feed provider when running this test?
• What instrument(s) (and expiry if applicable) have you selected?
• What Data Series Type have you selected? Example: Tick, Minute, Day
• What From and To date is selected?
• If you open a chart for the same instrument(s) and the same date range, then right-click the chart and select 'Reload all historical data' is the historical data showing on the chart?
• Is your strategy a multi-instrument or multi-time frame strategy?
• Do you receive an error on the screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

[Berry Trader]

Hi Gaby,
Here are the answers to your questions:

• What version of NinjaTrader are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.?.?.?)

8.1.3.1 64-bit

• Do you see similar results when running the same test on the SampleMaCrossOver strategy in NinjaTrader with the same settings as your strategy?

The SampleMACrossOver Strategy works fine in the analyzer

• Who are you connected to? This is displayed in green in the lower-left corner of the Control Center window.

Rithmic

• Are you connected to your data feed provider when running this test?

Yes

• What instrument(s) (and expiry if applicable) have you selected?

I selected NQ 03-25, This is the same From and To that I used with the Sample MA Cross

• What Data Series Type have you selected? Example: Tick, Minute, Day

Last, Tick, 100. This is the same From and To that I used with the Sample MA Cross

• What From and To date is selected?

From 1/20/2025 to 1/24/2025. This is the same that I used with the Sample MA Cross

• If you open a chart for the same instrument(s) and the same date range, then right-click the chart and select 'Reload all historical data' is the historical data showing on the chart?

Yes, and I have clicked that option.

• Is your strategy a multi-instrument or multi-time frame strategy?

Yes, My strategy is multi-instrument and multi-timeframe. It expects index[0] to be NQ 03-25 (Last. Tick, 100).
It additionally loads these data series (TickP =100):
AddDataSeries("ES 03-25", Data.BarsPeriodType.Tick, TickP, Data.MarketDataType.Last);
AddDataSeries("NQ 03-25", Data.BarsPeriodType.Tick, TickP*10, Data.MarketDataType.Last);
AddDataSeries("ES 03-25", Data.BarsPeriodType.Tick, TickP*10, Data.MarketDataType.Last);

• Do you receive an error on the screen? Are there errors on the Log tab of the Control Center? If so, what do these errors report?

When I load my script into the strategy analyzer, no errors appear on the screen. No errors appear in the log file.


My strategy runs without any errors (on screen or in the log file) during play back and during real time.

[NinjaTrader_Gaby]

Hello BerryTrader,

AddDataSeries() is not supported to be used dynamically. All arguments supplied to AddDataSeries() should be hardcoded.

This is noted on the Help Guide page for AddDataSeries():

"Arguments supplied to AddDataSeries() should be hardcoded and NOT dependent on run-time variables which cannot be reliably obtained during State.Configure (e.g., Instrument, Bars, or user input). Attempting to add a data series dynamically is NOT guaranteed and therefore should be avoided."

https://ninjatrader.com/support/helpGuies/nt8/adddataseries.htm

First remove the unsupported code from the script and test again.

[Berry Trader]

Thank you Gaby,
The Variable holding the the number of ticks for the dataseries, "TickP," was set as a parameter, available from the strategy analyzer screen.
I thought the point of the strategy analyzer was to test various input variables without having to change the script every time.
I'll change all my strategies to hard code the AddDataSeries from now on.

I changed my data series statements to this:
AddDataSeries("ES 03-25", Data.BarsPeriodType.Tick, 100, Data.MarketDataType.Last);
AddDataSeries("NQ 03-25", Data.BarsPeriodType.Tick, 1000, Data.MarketDataType.Last);
AddDataSeries("ES 03-25", Data.BarsPeriodType.Tick, 1000, Data.MarketDataType.Last);

It still runs without screen error and without log errors without taking any trades that it does during playback over the same time period.

Also the link you provided above gives me a 404 error, but I was able to search https://ninjatrader.com//support/helpGuides/nt8/
with the search word AddDataSeries and found the page you were suggesting.

Thank you, and do you have any suggestions or questions?

[NinjaTrader_Gaby]

You can use the analyzer to test different input variables, however these input variables shouldn't be used in an AddDataSeries() call.

Please review the help guide page on the differences on real-time vs backtest.

https://ninjatrader.com/support/helpGuides/nt8/discrepancies_real-time_vs_bac.htm

A strategy running real-time (live brokerage account, live market simulation, Playback connection etc...) will produce different results than the performance results generated during a backtest (Strategy Analyzer).

Live orders are filled on an exchange with a trading partner on an agreed upon price based on market dynamics. Backtest orders are not using these market dynamics. Instead these are filled based on logical rules from processing historical data.

• When in historical data, only the Open, High, Low, and Close will be available and there will be no intra-bar data.
  • This means actions cannot happen intra-bar, fills cannot happen intra-bar. All prices and actions come from and occur when the bar closes as this is all the information that is known
  • Because of this, OnBarUpdate will only update 'On bar close' as it does not have the intra-bar information necessary for 'On price change' or 'On each tick' and the script will not have the intra-bar information to accurately fill an order at the exact price and time.

Intra-bar granularity adds a second data series such as a 1 tick series using AddDataSeries() so that the strategy or indicator has the individual ticks in the historical data in between the High and Low of the primary series.

• This allows for more accurate trades by supplying the correct price at the correct time for the order to fill with. The orders placed will need to use the BarsInProgress (BarsArray) index of that finer resolution series.
  • https://drive.google.com/file/d/12F6...C3TgPC3pA/view
  • Help guide link

In NinjaTrader 8, there have been two new enhancements so that programmers may not have to manually add this secondary series and code the script for high accuracy fills (Order Fill Resolution) or for intra-bar actions (TickReplay) depending on the needs of the script.
Note: bar types that are IsRemoveLastBarSupported cannot be used with TickReplay and generally cause inaccurate results when backtesting in historical data.
Note: High Order Fill Resolution allows for intra-bar order fills with historical processing, but is not compatible with Tick Replay.
Please see this support article for a detailed explanation:


https://support.ninjatrader.com/s/ar...ar-granularity

[Berry Trader]

Thank you Gabby, I will study these and repost here when I can formulate a useful question. For now, my script runs well in real time and during replay. I am hoping to find a way to move through a series of days more quickly that watching the replay function@ 10x. Does replay work with the same accuracy, regardless of the chosen playback speed? Am I likely to get similar results if I play back at 1x, 50x, MAX...?
Berry

[NinjaTrader_Gaby]

Hello Berry Trader,

Yes, the replay will generate the same results regardless of playback speed.