Historical back testing doesn't fill until next bar

[dweems]

I have my strategy set to define my orders on the first tick of a bar. I see the orders being set right away as desired but even though the bar overlaps the entry prices I define, they are never filled until the next bar if that next bar overlaps the price. Given the 3-step process for order backfill, it seems like if the bar overlaps my entry price, it should fill. But it never does.
And, unfortunately I can't try Tick Replay because my strategy analyzes both the bid and ask prices and apparently it can't handle those, only the last price.
Any suggestions?

[NinjaTrader_ChelseaB]

Hello dweems,

To understand the issue, it will be necessary to print (or write to file) the order.ToString() in OnOrderUpdate() and the bar information in OnBarUpdate().

The example linked below provides sample code of writing information to file for analysis.

https://support.ninjatrader.com/s/article/Real-time-Replay-Historical-Comparisons-Test

As well as a link to a support article on using prints to understand behavior.

https://support.ninjatrader.com/s/article/Developer-Guide-Debugging-using-Print-and-TraceOrders

If no intra-bar granularity is implemented, orders will fill if the limit or stop price is between the high and low of the bar.

In Backtest, Calculate will always be OnBarClose unless TickReplay is enabled.

Use Bars.GetAsk(CurrentBar) / Bars.GetBid(CurrentBar) to get the ask and bid in historical when using TickReplay.

[dweems]

I can see how those would help. Before I do that, this is some of the print out I create for tracking that I write to the output window. The lines starting with "auto" are mine. You will also see others that are automatically written by NinjaTrader when I send output to the output window. On my lines, BP = buy price, SP = sell price, LS = long stop, SS = short stop, RE = re-entry price and ML = max loss (not used here). These lines are the result of a call I make immediately after IsFirstTickOfBar within OnBarUpdate that analyzes the previous bar to make the decision to enter a trade or to handle trailing stops to exit a trade. If you see both buy/sell price along with a long/short stop price, those would result in a trade reversal at the price if triggered. The re-entry price is to re-enter a trade in the same direction if a trade exits prematurely. Other lines indicate "{DateTime} Long/Short ordered entered/exited at {price}". I write those when an order enters OnExecutionUpdate. Also I have provided a screenshot of the bars so you should see what I'm talking about. If this doesn't help I'll try to write the bar info to a file as you indicate.

auto 6/21/2024 2:02:00 AM BP: 0.00000 SP: 0.00000 LS: 0.00000 SS: 0.00000 LT: 0.00000 RE: 0.00000 ML: 0.00000
auto 6/21/2024 2:03:00 AM BP: 1.07176 SP: 1.07164 LS: 0.00000 SS: 0.00000 LT: 0.00000 RE: 0.00000 ML: 0.00000
6/21/2024 2:03:00 AM Strategy 'dcwAutoTrader25/331051247': Entered internal SubmitOrderUnmanaged() method at 6/21/2024 2:03:00 AM: BarsInProgress=1 Action=Buy OrderType=StopMarket Quantity=1,000 LimitPrice=0 StopPrice=1.0717'6 SignalName='Long Entry'
6/21/2024 2:03:00 AM Strategy 'dcwAutoTrader25/331051247': Entered internal SubmitOrderUnmanaged() method at 6/21/2024 2:03:00 AM: BarsInProgress=1 Action=SellShort OrderType=StopMarket Quantity=1,000 LimitPrice=0 StopPrice=1.0716'4 SignalName='Short Entry'
6/21/2024 2:03:00 AM Long order entered at 1.07176
auto 6/21/2024 2:04:00 AM BP: 0.00000 SP: 1.07164 LS: 1.07164 SS: 0.00000 LT: 0.00000 RE: 1.07177 ML: 0.00000
6/21/2024 2:04:00 AM Strategy 'dcwAutoTrader25/331051247': Entered internal SubmitOrderUnmanaged() method at 6/21/2024 2:04:00 AM: BarsInProgress=1 Action=Sell OrderType=StopMarket Quantity=1,000 LimitPrice=0 StopPrice=1.0716'4 SignalName='Long Exit Stop'
auto 6/21/2024 2:05:00 AM BP: 0.00000 SP: 1.07164 LS: 1.07164 SS: 0.00000 LT: 0.00000 RE: 1.07182 ML: 0.00000
auto 6/21/2024 2:06:00 AM BP: 0.00000 SP: 0.00000 LS: 1.07179 SS: 0.00000 LT: 0.00000 RE: 1.07197 ML: 0.00000
6/21/2024 2:06:00 AM Strategy 'dcwAutoTrader25/331051247': Changed custom managed order: orderId='NT-00002-5' account='Sim101' name='Long Exit Stop' orderState=Working instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.07164 quantity=1,000 tif=Gtc oco='EURUSD _0128152224Long Exit Stop' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2024-06-21 02:04:00' gtd='2099-12-01' statementDate='2024-07-01' limitPriceChanged=0 quantityChanged=1000 stopPriceChanged=1.07179
6/21/2024 2:06:00 AM Strategy 'dcwAutoTrader25/331051247': Changed custom managed order: orderId='NT-00001-5' account='Sim101' name='Short Entry' orderState=Working instrument='EURUSD' orderAction=SellShort orderType='Stop Market' limitPrice=0 stopPrice=1.07164 quantity=1,000 tif=Gtc oco='EURUSD _0128151494Short Entry' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2024-06-21 02:03:00' gtd='2099-12-01' statementDate='2024-07-01' limitPriceChanged=0 quantityChanged=1000 stopPriceChanged=0
auto 6/21/2024 2:07:00 AM BP: 0.00000 SP: 0.00000 LS: 1.07184 SS: 0.00000 LT: 0.00000 RE: 1.07197 ML: 0.00000
6/21/2024 2:07:00 AM Strategy 'dcwAutoTrader25/331051247': Changed custom managed order: orderId='NT-00002-5' account='Sim101' name='Long Exit Stop' orderState=Working instrument='EURUSD' orderAction=Sell orderType='Stop Market' limitPrice=0 stopPrice=1.07179 quantity=1,000 tif=Gtc oco='EURUSD _0128152224Long Exit Stop' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2024-06-21 02:04:00' gtd='2099-12-01' statementDate='2024-07-01' limitPriceChanged=0 quantityChanged=1000 stopPriceChanged=1.07184
6/21/2024 2:07:00 AM Long order exited at 1.07184

[NinjaTrader_ChelseaB]

Hello dweems,

Also print the High and Low of the bar, so we can see what bar information the script is processing.

Note, that without TickReplay Calculate will always be OnBarClose and orders will always be submitted after the bar closes, which will fill on the next bar.

[dweems]

Your last note seems to confirm what I'm seeing. I'm making my orders immediately after IsFirstTickOfBar but if I understand correctly what you are saying, my new orders won't be executed until OnBarClose executes and therefore the orders will not be filled until the next bar. So what I need to do is create my new orders OnBarClose. But I don't see that as available to the programmer. I only see IsFirstTickOfBar. In truth, OnBarClose would be the ideal time for me to evaluate the data and make my new orders. Is there a mechanism I can do that? From your comments and from reading, I'm presuming this is not a problem with real time data or with Tick Replay. But as noted earlier, I access the bid and ask data rather than last data and find that I can't access Tick Replay when I access bid/ask data. So am I forced to do all my analysis on real time data? Or will I run into a similar situation even there?

[NinjaTrader_ChelseaB]

Hello,

"I'm making my orders immediately after IsFirstTickOfBar"

Without TickReplay Calculate is OnBarClose only in historical and IsFirstTickOfBar will always be true and will never be false.
IsFirstTickOfBar is only informative with Calculate.OnEachTick or Calculate.OnPriceChange in real-time or with TickReplay enabled.

https://ninjatrader.com/support/helpGuides/nt8/isfirsttickofbar.htm

However, if you were placing an order on the first tick of a new bar, this would be basically the same as OnBarClose, except that the condition bar is the previous bar instead of the new bar that just opened being the condition bar.

The script will be using OnBarClose already if TickReplay is not enabled. The Calculate setting can be assigned in State.Configure to be effective when backtesting.