Tick data in higher resolution is off by 1 bar of the main data series

[kreztmok]

Dear all,

Tick data in higher resolution is off by 1 bar of the main data series (i.e. Tick OHLC show OHLC of the next minute bar instead of current minute bar)

I think it is because the first 1 min bar is from 00:00 to 00:59 and is shown as 00:01 while tick data starts from 00:00.

My logic in higher resolution is using the next minute bar tick data instead of current minute bar tick data, which throws everything off.

Does someone has any idea on how to solve this?

Greetings,
Kreztmok

[NinjaTrader_Emily]

Hello Kreztmok,

Thank you for your post and welcome to the NinjaTrader forum community!

How are you accessing the OHLC data in your script? What series is your main data series, and what series are you adding for tick data? If you specify the BarsInProgress you would like OHLC data for or use the Opens/Highs/Lows/Closes series for the specific barsInProgressIndex, does this resolve the discrepancy? For more info:

• https://ninjatrader.com/support/help...inprogress.htm
• https://ninjatrader.com/support/help...riceseries.htm
  • See the list under "Multi-Time Frame ISeries<double>"

We have a full informative page about multi-series scripts here:

• https://ninjatrader.com/support/help...nstruments.htm
  • The section titled "How Bars Data is Referenced" shows which data is known depending on whether the calculation is on bar close or on price change/on each tick.

I look forward to your reply.

[kreztmok]

Hey Emily,

Thank you for the prompt response.

I'm accessing the OHLC using:
if (BarsInProgress == 1)
so i'm sure my logic is handled with the tick data series.

Primary data series is the input in the Strategy Analyzer, for my case the 1 minute
.
Second data series is tick and added like this:
AddDataSeries(BarsPeriodType.Tick,1);

i have written the following logic to catch this discrepancy:

currentbar = CurrentBars[0];
if (BarsInProgress == 0 && currentbar>=78)
{
System.Diagnostics.Debug.WriteLine(Open[0] + ", " + Low[0] + ", " + High[0] + ", " + Close[0] + " 1 min data at time: 0"+ToTime(Time[0]));
}

if (BarsInProgress == 1 && currentbar>=78)
{
System.Diagnostics.Debug.WriteLine(Open[0] + ", " + Low[0] + ", " + High[0] + ", " + Close[0] + " 1 tick data at time: 0"+ToTime(Time[0]));
}

this is the output:
2024-02-15 00:00:08:650 4798, 4797.25, 4798.25, 4797.25 1 min data at time: 011900
2024-02-15 00:00:26:221 4797.25, 4797.25, 4797.25, 4797.25 1 tick data at time: 011901

The next 1 minute bar opens at 4797.25, so the tick data is referencing this bar.

I have no idea why the Tick data series is not referencing the current bar of the primary data series.

[NinjaTrader_Emily]

Hello kreztmok,

Thank you for your reply.

Rather than System.Diagnostics.Debug.WriteLine, you could use the Print() method to generate output to the NinjaScript Output window:

https://ninjatrader.com/support/helpGuides/nt8/print.htm

With that said, you stated, "The next 1 minute bar opens at 4797.25, so the tick data is referencing this bar."
4797.25 is also the Close price of the 1 minute bar per the output:
2024-02-15 00:00:08:650 4798, 4797.25, 4798.25, 4797.25 1 min data at time: 011900

Additionally, a bar closure is determined when the first tick of a new bar is generated. It is very common for the close price of one bar to be the same value as the open price of the next bar. For more information:

• https://ninjatrader.com/support/help...ttickofbar.htm
  • As noted on that page: "In NinjaTrader's event driven framework, bar closures are signaled by the tick that opens the next bar. The price of the last tick of a bar can be referenced by checking Close[1] on IsFirstTickOfBar."

To trigger the output from the tick series, you check the following in your condition:
if (BarsInProgress == 1 && currentbar>=78)

What is the value if you print the CurrentBar of the tick series, or CurrentBars[1]? You could even print out CurrentBars[0] and CurrentBars[1] to see the difference between what bar is being printed for each series.

https://ninjatrader.com/support/helpGuides/nt8/currentbar.htm

https://ninjatrader.com/support/helpGuides/nt8/currentbars.htm

"I have no idea why the Tick data series is not referencing the current bar of the primary data series."
The tick series is a separate data series that has incoming data agnostic to the primary data series; if you print OHLC data for 0 barsAgo from the tick series, that will be data from the current bar in the tick series and not data specific to the 1-minute series. I do not understand what you mean by having the tick seris "referencing the current bar of the primary data series." You are working with your variable, currentbar, which is set per the following line:
currentbar = CurrentBars[0];

As long as that value for currentbar is greater than or equal to 78 you are generating the output. That value is not tied to the CurrentBar value of the tick series whatsoever. Please add the CurrentBar values to your prints. For more information and tips on creating informative debugging prints:

https://support.ninjatrader.com/s/article/Developer-Guide-Debugging-using-Print-and-TraceOrders?language=en_US

Please let me know if I may be of further assistance.

[kreztmok]

Hey Emily,

Thanks for the response.

I use System.Diagnostics.Debug.WriteLine because I'm debugging in Visual Studio.


In this case the close price of the 1 min bar is same as open of next bar; 4798.00

I have added these two lines within the if statement:

currentbar = CurrentBars[0];
System.Diagnostics.Debug.WriteLine(currentbar);

currentbar1 = CurrentBars[1];
System.Diagnostics.Debug.WriteLine(currentbar1);

Output:
2024-02-15 16:07:17:121 78
2024-02-15 16:07:20:105 3388

"The tick series is a separate data series that has incoming data agnostic to the primary data series"

I thought that any higher resolution series like ticks is connected to the primary series.
So that the open, high, low or close data is the same as the open tick, highest, lowest and close tick within that same 1 minute bar.

[NinjaTrader_Emily]

Hello kreztmok,

Thank you for your reply.

I suggest reviewing the page about multi-time frame and multi-instrument scripts for more information about how added series work:

https://ninjatrader.com/support/helpGuides/nt8/multi-time_frame__instruments.htm

There are also some helpful warnings and tips on the AddDataSeries() page of the help guide:

https://ninjatrader.com/support/helpGuides/nt8/adddataseries.htm

"I thought that any higher resolution series like ticks is connected to the primary series.
So that the open, high, low or close data is the same as the open tick, highest, lowest and close tick within that same 1 minute bar."

This is not the case. Simply obtaining the OHLC values from the 1-minute series would return the values you are referring to. An added 1-tick data series would generate a new bar for each incoming tick. If you want to get tick-by-tick updates for the 1-minute series you could set your script to calculate On Each Tick instead of On Bar Close or On Price Change. For more information regarding the Calculate property, which determines how frequently OnBarUpdate() is called:

https://ninjatrader.com/support/helpGuides/nt8/calculate.htm

Please feel free to reach out with any additional questions or concerns.

[kreztmok]

Hi Emily,

I already had the strategy calculate on Each Tick.

So if the data series are not connected, I have to find a way to synchronize the tick with the minute series.
I will take a look and see how I can resolve this.

Anyways, thanks for the support.

[kreztmok]

Hello Emily,

I solved the problem of the tick series not being in sync with the min series by counting the total ticks of the previous minute bar and then referencing this in my logic.

private int tick_count =0;
private int Lastbar_tick_count =1;

if (BarsInProgress == 0)
{
Lastbar_tick_count = tick_count;
tick_count =0; // reset counter on new bar
}

if (BarsInProgress == 1)
tick_count++; // increment counter


if (Lastbar_tick_count==0)
{
Lastbar_tick_count=0;
}


if (BarsInProgress ==1)
Lastbar_tick_count++;

I then use this in my logic like this:

Opens[1][Lastbar_tick_count]

Now the tick series is referencing the current minute series.

One error i don't seem to understand the cause of is:
Ignored SubmitOrderUnmanaged() method at 17/01/2024 01:19:01: BarsInProgress=1 Action=SellShort OrderType=StopLimit Quantity=1 LimitPrice=4797.50 StopPrice=4797.25 SignalName='Short limit entry' FromEntrySignal='' Reason='Invalid order price, please see log tab'

It says the order is ignored because the current tick price is below or equal to the StopPrice.
However, the Stop price (4797.25) is well below my current tick price (4798), which should not trigger this error.

How can I solve this problem?

Many thanks.

[NinjaTrader_Emily]

Hello kreztmok,

Thank you for your reply.

What is the full message from the log tab for this order?

• To send a screenshot with Windows 10 or newer I would recommend using the Windows Snipping Tool.
  • Click here for instructions
• Alternatively to send a screenshot press Alt + PRINT SCREEN to take a screenshot of the selected window. Then go to Start--> Accessories--> Paint, and press CTRL + V to paste the image. Lastly, save it as a jpeg file and send the file as an attachment.
  • Click here for detailed instruction

I look forward to your reply.

[kreztmok]

Hello Emily,

Below the ss:

T
his is the full text:

17/02/2024 13:12:52	Default	Strategy 'UnmanagedTemplate/-1': An order has been ignored since the stop price ‘4797.25’ near the bar stamped ‘01/17/2024 01:19:01’ is invalid based on the price range of the bar. This is an invalid order and subsequent orders may also be ignored.

[NinjaTrader_Emily]

Hello kreztmok,

Thank you for your reply.

This message has to do with how historical bars are built and broken down into virtual bars based on the OHLC prices. The price of the order was ignored due to the virtual bar and the stop price being on the wrong side of the market based on the virtual bar. We have a breakdown of the historical fill algorithm in the help guide here:

https://ninjatrader.com/support/helpGuides/nt8/understanding_historical_fill_.htm

This has also been discussed in other threads, such as the following:

https://forum.ninjatrader.com/forum/ninjatrader-8/strategy-development/1102709-understanding-backtest-stop-loss-error-message?p=1104682#post1104682

https://forum.ninjatrader.com/forum/ninjatrader-8/platform-technical-support-aa/1132163-order-ignored-stop-price-12364-75-near-the-bar-stamped-2020-12-09-7-50-00-pm?p=1132193#post1132193

Since you are already adding a 1-tick data series, one way to prevent this and introduce more granular data for the historical fill algorithm is to submit orders to the 1-tick series. We have a reference sample demonstrating how to backtest with granularity here:

https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

Please let us know if we may be of further assistance.

[kreztmok]

Hi Emily,

Thanks for the links, very helpful.

I believe to have found the cause why my orders are getting ignored.

Before I submit orders I check for Shorting if the price is above my Stop price, if yes the order gets submitted if (Opens[1][Lastbar_tick_count] <= Stop_price) .

However, because my tick series is not in sync with my minute series I have to go back 1 min bar (in ticks).
However, the order is submitted to the not in sync tick series. That's why I still get the order ignored error.

Is there anyway I can add a modified tick series so that the order gets submitted to the right current price?

Greetings,
Kreztmok

[NinjaTrader_Emily]

Hello Kreztmok,

Thank you for your reply.

Before I submit orders I check for Shorting if the price is above my Stop price

Which price are you checking to see if it is above your stop price? It seems you are checking the Open price from the tick series and you said you have to "go back 1 min bar (in ticks)" which I do not understand. Are you not able to simply check the desired Open price from the primary 1-minute series? You are still saying that the tick series is not in sync with the minute series - what do you mean that it is not in sync? If possible, please demonstrate what you are referring to as "not in sync" with some print statements and the output from those prints so I may better understand and assist you.

Is there anyway I can add a modified tick series so that the order gets submitted to the right current price?

What kind of "modified" tick series are you referring to? I am not sure what you mean. The reference sample I posted demonstrates how to submit orders to the added series using the overload that allows you to specify the barsInProgressIndex on which you would like the order placed. Have you modified your order methods to specify that the orders should be submitted to the 1-tick series and not the 1-minute series?
Here is the link to that strategy sample again:

https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

For additional explanation of this concept, please see "Working with a Multi-Instrument Strategy" on the following page:

https://ninjatrader.com/support/helpGuides/nt8/advanced_order_handling.htm

I look forward to your clarification.

[kreztmok]

Hi Emily,

I think it's best to go back to the root-cause of the problem.

The downloaded data from My Ninjatrader data provider, gives the first minute bar of a day a timestamp of 00:01:00, while the first timestamp of the tick series is 00:00:00.
The result is, when i try to access bar 3 on the minute time frame this is considered to have a timestamp of 00:03:00 and on the tick series a timestamp of 00:02:00.

How can i adjust the timestamps of my downloaded data so that the tick series also starts with timestamp 00:01:00?


Thank you.

[NinjaTrader_Emily]

Hello kreztmok,

Thank you for your reply.

I don't think you are understanding the timestamps of the data and how tick bars vs. minute bars are built. You may not change the timestamp of your data; most data is natively timestamped by your data provider and sent to NinjaTrader already timestamped. This is noted in the "Real-Time Timestamp" column on the following page:

• https://ninjatrader.com/support/help...y_provider.htm

Further, when it comes to how bars are built, theya re timestamped with the closing time of the bar. This is mentioned on the page that describes how bars are built:

• https://ninjatrader.com/support/help..._are_built.htm
  • Also note the underlying base data types; there are three data types which are Tick, Minute, and Daily

The first minute with a timestamp of 00:01:00 contains data from 00:00:00 (midnight) to 00.00.59 (12:00:59 am). If you were to break the minute down into separate ticks worth of data, it could potentially contain several ticks. The Open, High, Low, and Close values from that minute would be the opening tick (the same one you are seeing at 00:00:00), the highest tick during that minute, the lowest tick during that minute, and the last tick of that minute would be the Close price.
The first tick with a timestamp of 00:00:00 contains one tick's worth of data that is the first tick at midnight. It is one data point where the Open, High, Low, and Close are all the same value from that one tick.

Below is a link to a video that hopefully demonstrates this visually as well. I plotted 1-tick candles on top of a 1-minute box chart. The red/green boxes represent each 1-minute bar, and the ticks all appear as little dashes throughout the box. Tick candles appear as dashes because they are one singular data point with the same OHLC for that one candle. As the ticks go up and down, you can see how the 1-minute box adjusts to the Open, High, Low, and Close based on the ticks of incoming data:

• https://www.screencast.com/t/yooQv22p00

No set amount of ticks goes into making a 1-minute bar. You could have a slow-moving market where each minute only has 1 tick of movement, or you could have an extremely volatile market where hundreds or thousands of ticks occur within the same minute. The CurrentBar index will not match between minute and tick data. For example, in the following screenshot with the Data Box open, you can see that the "Bar index" for the minute data does not match the "Bar index" for the tick data:

https://www.screencast.com/t/YzAHJptqPv

I hope this helps to clarify. Please feel free to reach out with any additional questions or concerns.