Huge discrepancy between live and backtest

[samish18]

Running a strategy live and backtest produces different results. I am executing onBarClose and have added a secondary data series of 1-tick to improve accuracy. The times when the strategy takes trades differs slightly, but the price fills are very different. I am attaching both the live and backtest results

[NinjaTrader_ChelseaB]

Hello samish18,

Are you using the BarsInProgress index of the added 1 tick series for the order method calls?
EnterLong(int barsInProgressIndex, int quantity, string signalName)

What is the BarsInProgress index you are supplying to the order methods?
During what BarsInProgress is the order submitted?

To understand what is causing differences write the price data and condition values to a text file, write the order.ToString() and execution.ToString() in a text file.
Run the strategy again in real-time and backtest.

Below is a link to a support article that provides instructions and an example script with sample code.

https://support.ninjatrader.com/s/article/Developer-Guide-Comparing-Strategy-Results

Attach the output text files to your reply and I will be happy to assist with analyzing the output to ensure the first bar and last bar of the data is the same, the conditions are evaluating the same, and compare the order fills.

[samish18]

Yes, I am using the 1 barsIndex for the 1 tick added dataseries

[samish18]

I compared the printed closes from the 1 tick between live and backtest. This is terrible! full point plus differences between live and backtest! Attached is the results, I copied to live prints to the google doc

[NinjaTrader_ChelseaB]

Hello samish18,

Likely the data is different.

Once you have written to text file the OHLC of the bar, the order.ToString(), and execution.ToString() from the example script I have provided you and attached these output files to your post, I will be happy to assist with analyzing what is causing differences.

Be sure you have watched the 'Comparing Real-Time Performance with Historical or Playback Data' to understand what is being written to file with the C# StreamWriter.

[samish18]

Attached are the prints from the livetest and the backtest

[NinjaTrader_ChelseaB]

Hello samish18,

May I confirm you have watched the 'Comparing Real-Time Performance with Historical or Playback Data' to understand what is being written to file with the C# StreamWriter.

May I confirm you are using the example I've provided you and modeling your StreamWriter strings the same way?

We are looking for the OHLC of the bar, the order.ToString(), and the execution.ToString(). The output appears to show the order.ToString(), but I'm not seeing the files for the bar data and execution information.

You can copy and paste these from the example script to save time.

[samish18]

I did not see where to print execution.ToString, is that in the same section as order.ToString. The secondary series is a 1 tick for accuracy, where open, high , low close and equal. I am attaching a screenshot of the chart, it appears running live is executing one bar behind what happens in the backtest

[NinjaTrader_ChelseaB]

Hello samish18,

I just want to confirm, are you copying and pasting the StreamWriter code from the example script?

[samish18]

I am using Print(Time[0] + " " +Closes[1][0]). What do you make of the live strategy lagging the backtest by 1 bar?

[NinjaTrader_ChelseaB]

Hello samish18,

I would assume that Closes[1] is a different bar type or interval or different instrument than Closes[0].

I think you are confirming you have not reviewed the support article or watched the video or reviewed the example script in post # 2, is this correct?

As you are not wanting to review the information or provide the requested information, I would recommend contacting a professional NinjaScript Consultant who would be eager to create or modify this script at your request or assist you with your script. The NinjaTrader Ecosystem has affiliate contacts who provide educational as well as consulting services.

You can search our extensive library of NinjaScript consultants through the link below. Simply enter a consultant name or search by using our filter categories. Once you have identified your consultants of choice, please visit each consultant's site for more information or contact them directly to learn more!
Programming Services - https://ninjatraderecosystem.com/sea...mming-services
Educators - https://ninjatraderecosystem.com/sea...ures=education

You can locate the contact information for the consultants on their direct websites for any additional questions you may have. Since these consultants are third party services for NinjaTrader all pricing and support information will need to be obtained through the consultant.

This NinjaTrader Ecosystem website is for educational and informational purposes only and should not be considered a solicitation to buy or sell a futures contract or make any other type of investment decision. The companies and services listed on this website are not to be considered a recommendation and it is the reader's responsibility to evaluate any product, service, or company. NinjaTrader Ecosystem LLC is not responsible for the accuracy or content of any product, service or company linked to on this website.

Please let me know if you have any questions, or if I can provide any further assistance.

[samish18]

Closes[1] is the same instrument but at a 1-tick level for improved backtest accuracy.

The link you sent, Developer Guide - Comparing Strategy Results (ninjatrader.com), does not include any information on execution.toString(). Is StreamWriter a better option to simply printing from the strategy? I am under the impression that they generate the same result.

Do you know why the strategy running live lags executions by a full bar of the backtest - is this standard?

[NinjaTrader_ChelseaB]

Hello samish18,

The execution.ToString() is written to text file on line 167 and 168 in the RealtimeReplayHistoricalComparisonsTest example script.

StreamWriter writes all of the information to file which can be too much for the NinjaScript Output window which has a maximum number of lines.

"Do you know why the strategy running live lags executions by a full bar of the backtest"

I don't know what you have coded in your script or what settings are being used to run this.

If the script is running with Calculate.OnEachTick or .OnPriceChange and TickReplay is not enabled, the script would be running with Calculate.OnBarClose in backtest and with Calculate.OnEachTick/.OnPriceChange in real-time.

If the script uses OnMarketData() and TickReplay is not enabled, the script could be reacting to real-time market changes in real-time and not in historical.

The script may not have properly implemented 1-tick intra-bar granularity.

The data may be different backtest than real-time, such as started on a different bar.

Without information, I would not be able to say what your script is doing.

A consultant may be able to do this process of watching video and studying the example script to add this debugging output to your script on your behalf.