Market Replay issues caused by FirstBarOfSession

[ATI user]

I have been chasing a problem with MR testing inconsistencies for some time now and have finally proved the problem not to be MR but rather how NT starts a session for a given chart type or interval.

If you have a strategy that uses a secondary data series as a filter/trigger, results will be affected by how many bars are loaded into the chart before starting MR..and maybe at what time the chart is started. Only 1 minute bar series seems to be reliable since other series may not start each session with a new bar (even 5 min...renko, vol, tick even worse). This throws off the filtering/triggering so you can not get consistent results from one computer to another or even for a one day test on the same computer. . A 1 min chart always starts a new session with a new bar. The problems in the attached pics were only solved when the secondary series was changed to 1 min.

I was told some weeks/months ago that a 'major' update to MR is planned. It will not solve this problem unless the underlying issue of starting a new session with a new bar regardless of chart type or interval is resolved.

Some months/years ago I had an issue with Range bars not starting each session with a new bar. NT changed this and its works fine now ...although I have not tested as a secondary data series filter.

[NinjaTrader_Bertrand]

Thanks much for the post ATI user - so if I follow correctly, the issue would be the reset of the bars building count for the added series is not working consistently for you in replay? Generally for a MultiSeries strategy it would be expected that OnBarUpdate() is only called if the BarsRequired is fulfilled for all series the strategy uses.

I will check into the bars building for a MultiSeries script (other than 1 minute) and see if I can reproduce here on our end.

Thanks again,

[NinjaTrader_Bertrand]

ATI user, what prints are you exactly comparing in your screenshots please?

Is there any way you could send me your strategy with proprietary logic removed so I could test / see your skeleton of add logic etc for reproducing?

I'm wondering though, all bars building counts should be reset at the session break - do you test perhaps with some continuous weekly session here as this happens?

[ATI user]

In the screenshots the pnl changes based on how many days loaded...this due to secondary data series filter difference for first bar of session

yes, I am running replay for several weeks of data most of the time...however, the difference still occurs just running one day of data when you try several 'days loaded' ....see pics for 1 min where with 6,8 and 10 days loaded the pnl is the same...however for non-1min pnl is off by thousands of dollars

[NinjaTrader_Bertrand]

Thanks ATI user, will continue to look into and see if we can create a scenario to reproduce - if you add Prints to your script for the FirstBarOfSession on the added series, would you see no prints in some instances if it would not running as 1 minute, correct?

[ATI user]

The problem is that, with the exception of 1 min dat series, more often than not the first tick of a bar does not occur as the first tick of the first bar of the session. For instance a 144 tick chart or 5 Renko chart will continue to build bars without starting a new bar on the first tick of the session. So if you use non-1min data series as secondary filters, results will vary because each time you enable a strategy, with varying number of days in replay and using FirstBarOfSession, you will be getting a different secondary series first bar of session which will though off all the bars throughout the test period relative to other tests with other days loaded. The only way to get consistent data in MR with multi-series charts is to base the sceondary series on 1 min so you always have the filter series start all of its bars on exactly the same tick regardless of how many days are loaded into the primary data series chart.

[NinjaTrader_Bertrand]

Thanks much for the clarification ATI user, looking into matters further and will get back to you on it.