Multi-Series Back-Testing Synchronization

[jflaggs]

Hello,

I am trying to run a back test with multiple series added and I've discovered that the CurrentBars count is not in sync even when the time interval is the same for all instruments. The indicator code below is applied to the instrument EURUSD 60min chart. I've included the output which shows that the current bar is dramatically out of sync even though all bars have a uniform period. How can I go about synchronizing the CurrentBar such that I can accurately back test?

if (State == State.Configure)
{
      // EURUSD 60 minute chart is BarsInProgress 0
      AddDataSeries("AUDUSD", Data.BarsPeriodType.Minute, 60, Data.MarketDataType.Last); // BarsInProgress 1
      AddDataSeries("USDCAD", Data.BarsPeriodType.Minute, 60, Data.MarketDataType.Last); // BarsInProgress 2
}

protected override void OnBarUpdate()
{
     var print = $"BIP: {BarsInProgress}, Current Bar: {CurrentBar}";
            
     Print(print);​
}

Actual Console Output:
[...]
BIP: 0, Current Bar: 1135
BIP: 1, Current Bar: 660
BIP: 2, Current Bar: 817
BIP: 0, Current Bar: 1136
BIP: 1, Current Bar: 661
BIP: 2, Current Bar: 818
[...]

Desired Console Output:
[...]
BIP: 0, Current Bar: 1135
BIP: 1, Current Bar: 1135
BIP: 2, Current Bar: 1135
BIP: 0, Current Bar: 1136
BIP: 1, Current Bar: 1136
BIP: 2, Current Bar: 1136
[...]

[NinjaTrader_ChelseaB]

Hello jflaggs,

It may be there is no historical data for one of the instruments for some periods.

If there were no trades for a short time for an instrument, that could mean no bar was produced during that time.

You could manually modify the the historical data and inject data that is not real if you want to close those gaps, should that be the issue.

https://ninjatrader.com/support/helpGuides/nt8/editing.htm

[jflaggs]

If that is the case then how can the CurrentBar be "almost" in-sync at the last bar?

The last few lines of the console output are:
BIP: 0, Current Bar: 10000
BIP: 1, Current Bar: 9999
BIP: 2, Current Bar: 9999

[NinjaTrader_ChelseaB]

Hello jflaggs,

Not enough information is provided.

You could print the time stamps of each bar to see what bars are being processed.

The entire output would be needed to for any analysis.

[jflaggs]

Here's more lines from the console. It seems its all for BIP 0. I've added the Time to the output. I've attached a file with more lines, but the file size limit is 1MB so I split into two files.

BIP: 0, Current Bar: 0, Time 6/10/2022 9:00:00 AM
BIP: 0, Current Bar: 1, Time 6/10/2022 10:00:00 AM
BIP: 0, Current Bar: 2, Time 6/10/2022 11:00:00 AM
BIP: 0, Current Bar: 3, Time 6/10/2022 12:00:00 PM
BIP: 0, Current Bar: 4, Time 6/10/2022 1:00:00 PM
BIP: 0, Current Bar: 5, Time 6/10/2022 2:00:00 PM
BIP: 0, Current Bar: 6, Time 6/13/2022 6:00:00 AM
BIP: 0, Current Bar: 7, Time 6/13/2022 7:00:00 AM
BIP: 0, Current Bar: 8, Time 6/13/2022 8:00:00 AM
BIP: 0, Current Bar: 9, Time 6/13/2022 9:00:00 AM
BIP: 0, Current Bar: 10, Time 6/13/2022 10:00:00 AM
BIP: 0, Current Bar: 11, Time 6/13/2022 11:00:00 AM
BIP: 0, Current Bar: 12, Time 6/13/2022 12:00:00 PM
BIP: 0, Current Bar: 13, Time 6/13/2022 1:00:00 PM
BIP: 0, Current Bar: 14, Time 6/13/2022 2:00:00 PM
BIP: 0, Current Bar: 15, Time 6/13/2022 3:00:00 PM
BIP: 0, Current Bar: 16, Time 6/13/2022 4:00:00 PM
BIP: 0, Current Bar: 17, Time 6/13/2022 5:00:00 PM
BIP: 0, Current Bar: 18, Time 6/13/2022 6:00:00 PM
BIP: 0, Current Bar: 19, Time 6/13/2022 7:00:00 PM
BIP: 0, Current Bar: 20, Time 6/13/2022 8:00:00 PM
BIP: 0, Current Bar: 21, Time 6/13/2022 9:00:00 PM
BIP: 0, Current Bar: 22, Time 6/13/2022 10:00:00 PM
BIP: 0, Current Bar: 23, Time 6/13/2022 11:00:00 PM
BIP: 0, Current Bar: 24, Time 6/14/2022 12:00:00 AM
BIP: 0, Current Bar: 25, Time 6/14/2022 1:00:00 AM
BIP: 0, Current Bar: 26, Time 6/14/2022 2:00:00 AM
BIP: 0, Current Bar: 27, Time 6/14/2022 3:00:00 AM
BIP: 0, Current Bar: 28, Time 6/14/2022 4:00:00 AM
BIP: 0, Current Bar: 29, Time 6/14/2022 5:00:00 AM
BIP: 0, Current Bar: 30, Time 6/14/2022 6:00:00 AM
BIP: 0, Current Bar: 31, Time 6/14/2022 7:00:00 AM
BIP: 0, Current Bar: 32, Time 6/14/2022 8:00:00 AM
BIP: 0, Current Bar: 33, Time 6/14/2022 9:00:00 AM
BIP: 0, Current Bar: 34, Time 6/14/2022 10:00:00 AM
BIP: 0, Current Bar: 35, Time 6/14/2022 11:00:00 AM
BIP: 0, Current Bar: 36, Time 6/14/2022 12:00:00 PM
BIP: 0, Current Bar: 37, Time 6/14/2022 1:00:00 PM
BIP: 0, Current Bar: 38, Time 6/14/2022 2:00:00 PM
BIP: 0, Current Bar: 39, Time 6/14/2022 3:00:00 PM
BIP: 0, Current Bar: 40, Time 6/14/2022 4:00:00 PM
BIP: 0, Current Bar: 41, Time 6/14/2022 5:00:00 PM
BIP: 0, Current Bar: 42, Time 6/14/2022 6:00:00 PM
BIP: 0, Current Bar: 43, Time 6/14/2022 7:00:00 PM
BIP: 0, Current Bar: 44, Time 6/14/2022 8:00:00 PM
BIP: 0, Current Bar: 45, Time 6/14/2022 9:00:00 PM
BIP: 0, Current Bar: 46, Time 6/14/2022 10:00:00 PM
BIP: 0, Current Bar: 47, Time 6/14/2022 11:00:00 PM
BIP: 0, Current Bar: 48, Time 6/15/2022 12:00:00 AM
BIP: 0, Current Bar: 49, Time 6/15/2022 1:00:00 AM
BIP: 0, Current Bar: 50, Time 6/15/2022 2:00:00 AM

[NinjaTrader_ChelseaB]

Hello jflaggs,

So this output is showing you that data for BIP 1 is missing from 6/10/2022 to 7/12/2022.
BIP 2 is missing data from 3/22/2024 2:00:00 PM to 4/30/2024 9:00:00 AM and from 6/1/2024 4:00:00 AM to 6/4/2024 1:00:00 AM
BIP 0 is missing data from 3/22/2024 2:00:00 PM to 5/20/2024 6:00:00 AM

This is just what I am seeing at a glance, there are likely more areas where some instruments have data and others do not.

[jflaggs]

I see, thank you for pointing that out. Now I understand why the CurrentBars are not aligned. In terms of backtesting, is there a way to ensure that I am processing the same "time slice" for all BIP each time OnBarUpdate is called? Either programmatically or with some NT8 settings.

For example, if BIP 1 is processing dates 3/22/2024 2:00:00 PM to 5/20/2024 6:00:00 AM, but BIP 0 or 2 has missing data, I would need to skip all this date range for BIP 0 and 2. In other words, I need to ensure that I only process data where all BIP have the same valid dates and skip the gaps where dates do not align.

[NinjaTrader_ChelseaB]

Hello jflaggs,

You could check the time of the other series is the same.

if (Times[0][0] == Times[1][0])

Or on a non-time based chart you could check that the bar times are within the same few minutes.

[jflaggs]

The issue with this approach is that you cannot compare indexes between two BIP series if there is missing data. Is there a more reliable way to ensure I'm operating on the same timestamp for all BIP series?

I'm essentially trying to calculate seriesC = seriesA + seriesB where the "+" operation should only apply to matching time stamps.

[NinjaTrader_ChelseaB]

Hello jflaggs,

The condition would only be evaluating if there is data at the same time for all series. If some series are missing data, the condition wouldn't evaluate as true.

If one series doesn't have the data, what do you want to do?
Do you want to calculate with the series that has data?
Do you want to modify the historical data in the Tools > Historical Data Window and add the missing bars with fake information?

[jflaggs]

If one series doesn't have the data, what do you want to do?

This is not the question I'm asking. I am asking the question, "If one series doesn't have the data, how will I know using indexing?"

Ex.

Series A (BIP 0) has daily dates 6/1/2024 - 6/20/2024 indexed A[0] through A[20]
Series B (BIP 1) has daily dates 6/1/2024 - 6/19/2024 indexed B[0] through B[19]

The CurrentBar property will be in sync up until the 19th, then Series B will be 1 bar behind A indefinitely.

In this case, the meaning of Times[0][0] == Times[1][0] will return false on 6/20/2024 since the comparison will evaluate to 6/20/2024 == 6/19/2024 -> FALSE.

So my question is not what to do when one series has data, but how to know when the series are out of sync using indexing? The only way I can think of to solve this issue is to index into both series A and B by date rather than by bars ago.

So something like this:

// MY QUESTION IS:
// Is there a way to implement this logic in NT8??​
if (Closes[0].IsValidAt(date) && Closes[1].IsValidAt(date))
{
    var value = Closes[0][date] + Closes[1][date];
}

This is very different from the logic you provided:

if(Times[0][barsAgo] == Times[1][barsAgo])
{
    var value = Closes[0][barsAgo] + Closes[1][barsAgo];
}

Hopefully that clarifies the issue I'm having.

[NinjaTrader_ChelseaB]

Hello jflaggs,

I wouldn't recommend using indexes. I would recommend comparing the date and time.

You could set a bool if at any point the CurrentBars[barsSeries index] is different, and then you would know at some point one series didn't have data, but there isn't any real correlation between added series. These are intended to be able to be different bar types and intervals and aren't designed to be synchronized.