Market entry order not filled in backtest on daily bars

[poseidon_sthlm]

Hello!

I'm testing a strategy on daily bars in the Strategy Analyzer and have encountered a strange situation where a market entry order doesn't get filled. (The situation occurs on a huge doji bar during extreme volatility.)

I use the managed approach with entry orders "Live until cancelled" and assign the entryOrder in OnOrderUpdate() to ensure the assignment occurs when expected according to "SampleOnOrderUpdate". The issue seems to be that the entry order gets stuck in orderState=Working and never reaches orderState = filled. No error message is thrown to the log.

How can I solve this issue?


The entry order is submited on the close of Friday 2020-03-20 and would be expected to be filled on the Open on Monday 2020-03-23.

++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
OnBarUpdate()
Time[0]: 2020-03-20 22:00:00 | CurrentBar: 306
++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
Position.MarketPosition: Flat
(entryOrder == null): True
2020-03-20 22:00:00 Strategy 'akBasicExitSignalsStrat/-1': Entered internal SubmitOrderManaged() method at 2020-03-20 22:00:00: BarsInProgress=0 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='LongBaseLine_306' FromEntrySignal=''

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OnOrderUpdate()
Time[0]: 2020-03-20 22:00:00 | CurrentBar: 306
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OHLC: 2409,75 | 2557,75 | 2318,75 | 2347,25
order.ToString(): orderId='NT-00099-568' account='Backtest' name='LongBaseLine_306' orderState=Submitted instrument='ES SEP23' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2020-03-20 22:00:00' gtd='2099-12-01' statementDate='2023-08-27'

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OnOrderUpdate()
Time[0]: 2020-03-20 22:00:00 | CurrentBar: 306
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OHLC: 2409,75 | 2557,75 | 2318,75 | 2347,25
order.ToString(): orderId='NT-00099-568' account='Backtest' name='LongBaseLine_306' orderState=Accepted instrument='ES SEP23' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2020-03-20 22:00:00' gtd='2099-12-01' statementDate='2023-08-27'

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OnOrderUpdate()
Time[0]: 2020-03-20 22:00:00 | CurrentBar: 306
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OHLC: 2409,75 | 2557,75 | 2318,75 | 2347,25
order.ToString(): orderId='NT-00099-568' account='Backtest' name='LongBaseLine_306' orderState=Working instrument='ES SEP23' orderAction=Buy orderType='Market' limitPrice=0 stopPrice=0 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2020-03-20 22:00:00' gtd='2099-12-01' statementDate='2023-08-27'

++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
OnBarUpdate()
Time[0]: 2020-03-23 22:00:00 | CurrentBar: 307
++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
Position.MarketPosition: Flat
(entryOrder == null): False
entryOrder.OrderState: Working

++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
OnBarUpdate()
Time[0]: 2020-03-24 22:00:00 | CurrentBar: 308
++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
Position.MarketPosition: Flat
(entryOrder == null): False
entryOrder.OrderState: Working

++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
OnBarUpdate()
Time[0]: 2020-03-25 22:00:00 | CurrentBar: 309
++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++
Position.MarketPosition: Flat
(entryOrder == null): False
entryOrder.OrderState: Working

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your post.

Please share a reduced exported copy of your strategy and the exact steps and settings you are using to reproduce the behavior so we may look into this matter further.

To export a copy of your strategy, go to Tools > Export > NinjaScript AddOn.

See the 'Exporting NinjaScript as Source Files' section of this help guide page for more information: https://ninjatrader.com/support/help...tAsSourceFiles

Note that a reduced copy refers to a copy of the script that contains the minimum amount of code needed to reproduce the issue. All other code is commented out or removed. To create a copy of your script to modify, open a New > NinjaScript Editor, select your script, right-click in the Editor, select 'Save as', name the script, and click OK.

I look forward to assisting further.

[poseidon_sthlm]

Hello again,

Here is a reduced strategy that can reproduce the issue described above.
MinStrat.zip

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your notes.

Please share the exact steps and settings you are using to reproduce the behavior using the script you shared as requested in post # 2 so we may look into this matter further.

If you are backtesting the script in the Strategy Analyzer and see the behavior occur, please send me a screenshot of all the Strategy Analyzer settings you are using so I may use those settings to test the script on my end.

If you are running the script in Playback, please let me know what data (Market Replay or Historical) you are using, instrument, dates you are testing on, what time/date you see the behavior occurs, and what strategy settings are being used to reproduce the behavior.

[poseidon_sthlm]

Hi,
I run the strategy in the Strategy Analyzer on ES-Sep23 daily bars with these settings ;



/Poseidon_sthlm

[NinjaTrader_BrandonH]

Hello poseidon_sthml,

Thanks for your notes.

This behavior could be caused by historical overfill occurring. Without using intra-bar granularity, NinjaTrader only knows if an order would have been filled (between the high and low) but not the order in which the orders will have filled as we only know the information of the finished bar.

This can result in multiple orders filling on the same bar which could result in an order reporting an OrderState of Working instead of being filled.

You could try implementing 1-Tick intrabar granularity in your script and test to see if the behavior continues to occur. Below is a link to a forum post on intra-bar granularity.

https://ninjatrader.com/support/foru...377#post773377

Please reference the SampleIntrabarBacktest example and the following Help Guide links for more information.

SampleIntrabarBacktest 'Backtesting NinjaScript Strategies with an intrabar granularity' - https://ninjatrader.com/support/helpGuides/nt8/backtesting_ninjascript_strate.htm

TickReplay — https://ninjatrader.com/support/help...ick_replay.htm

Developing for Tick Replay -
https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub= developing+for+tick+replay

Additional information may be found in this NinjaTrader Forum post —
https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/100192-comparing-real-time-historical-and-replay-performance?t=102504

[poseidon_sthlm]

Hi Brandon,

Thanks for your reply. I assume you can reproduce the situation I have described above using the strategy I attached.

I have now had the time to test my strategy for different years and instruments on daily bars and can confirm that the "overfill issue" is a notorious and frequent problem that occurs in almost all tests that I have done. Does this mean that it is not possible to use the NT Strategy Analyzer on daily bars (without a secondary time frame for intrabar granualrity)? I must say that would be an unexpected suprise to me. Right now I only have access to free daily data and my plan was to test some strategies on a large number of instruments on daily bars.

I understand the concept "over fill". But the scenario where an overfill would occur on a daily bar is highly unlikely. Besides,in the situation where the SL-order and target order may be filled on the same bar and one doesn't know which one is hit first, one would expect the SL to be filld as a precaution, it's also strange that no error message is thrown in this over-fill situation. Futher more I see a lot of trades on my chart wrere the entry and exit are filled intrabar and where there is no way for NT to know for sure if the targer or SL would be hit first. So the question remains why the over-fill problem occurs on some bars but not others?

I have tried to find a work around and cancel the entry order when it wasn't filled within two bars after the entry signal bar, but that didn't work. It seems that a market order cannot be cancelled with the CancelOrder() method. Could you suggest a work around to avoid this over-fill problem on daily bars (without using the unmanaged approach)?

Poseidon_sthlm

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your notes.

Strategies could be tested on Daily bars in the Strategy Analyzer window depending on the logic of the strategy but it is possible that historical overfill could occur when backtesting the strategy on Daily bars.

Without using intra-bar granularity, NinjaTrader only knows if an order would have filled (between the high and low) but not the order in which the orders will have filled as we only know the information of the finished bar. This can result in multiple orders filling on the same bar.

The workaround for this would be to add 1-Tick intra-bar granularity to the script, enabling Tick Replay, and submitting the orders to the added single tick series as noted in post # 6.

Otherwise, the best solution would be to use the Unmanaged Approach for the strategy and using OCOIDs. This would not be affected by Historical Overfills (even without intra-bar granularity or High Order Fill Resolution).

Unmanaged Approach: https://ninjatrader.com/support/help...d_approach.htm

[poseidon_sthlm]

Hi Brandon,

Thanks for your suggestions. I will try the unmanaged approcah for backesting.

/Poseidon_sthlm

[poseidon_sthlm]

Hi Brandon,

I have now modified my strategy according to the unmanaged approach. (This unmanaged strategy is attached.) The issue were a market entry order gets stuck in OrderState = Working (as described in post #1) has disappeared, although I have set the property IgnoreOverfill = false. Does this indicate that the reason behind the issue in the managed approach was not an overfill? If so, what could be the actual reason behind the unexpected behavior in the reduced strategy that I attached in post #3?

/Poseidon_sthlm

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your notes.

I am happy to hear you were able to resolve the behavior by using the unmanaged approach for order management.

Without adding intrabar granularity to the managed approach strategy and thoroughly debugging the strategy we would not be able to say for certain if the behavior was caused by historical overfill occurring or not. Debugging would have to be done on the managed approach strategy to understand exactly what was causing the behavior to occur.

Note that we do not offer debugging services in our support so it would be up to you to add the intrabar granularity to the script and debug the strategy's logic to understand how it is behaving.

[poseidon_sthlm]

Hello Brandon,

I have added an option to run my sttrategy with intrabar granularity. (The strategy is attached: MinStratIntrabar.zip ). With intrabar granualrity there is no issue were a market entry order gets stuck in OrderState = Working (as described in post #1) .

In summary, when I run my reduced stratgey in the Strategy Analyzer on daily bars:

1. with the managed approach, the entry order gets stuck in orderState=Working and never reaches orderState = filled.
2. with the managed approach and intrabar granularity, there is no issue
3. with the unmanaged approach when the property IgnoreOverfill = false, there is no Issue

Can we now rule out that the issue is due to historical over fills?

I still would prefer to run my strategies in the SA.on daily bars with the managed approach without intrabar granualrity. Could you analyze the cause behind this issue and maybe ask Development to fix it?

/Poseidon_sthlm

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your notes.

If you added intrabar granularity to your strategy and the behavior is not occurring when testing the strategy with intrabar granularity then yes, it was likely the historical overfill causing the behavior you reported.

Without using intra-bar granularity or High Order Fill Resolution, NinjaTrader only knows if an order would have filled (between the high and low) but not the order in which the orders will have filled as we only know the information of the finished bar.

This can result in multiple orders filling on the same bar.

We do have a feature request for improvements to be made with Historical Overfills without having to use intra-bar granularity or High Order Fill Resolution and this is tracked with ID# SFT-2778. I will add your vote to this request.

As with all feature requests, interest is tracked before implementation is considered, so we cannot offer an ETA or promise of fulfillment. If implemented, it will be noted on the Release Notes page of the Help Guide.

Release Notes — https://ninjatrader.com/support/help...ease_notes.htm

We recommend that if you plan to use Exit methods that you do not use Set methods in the same script, and vice versa (when using Set methods do not use Exit methods).

In real-time, when using the Managed Approach, (particularly, persons using the Strategy Builder with their own hard Exit logic while using Set methods for profit target/stop loss) an Exit that logically gets submitted while the Stop/Target fills will cause in-flight executions. If these exits fill partially, we can be left in an open position.

When back-testing, mixing Set methods with Exit methods can result in Historical Overfills where 3 orders, a stop, exit, and entry, are able to fill on the same bar.

Further, when using multiple exit methods it is always best to use intra-bar granularity or High Order Fill Resolution to prevent Historical Overfills.

Below is a link to a forum post on intra-bar granularity and High Order Fill Resolution.
https://ninjatrader.com/support/foru...377#post773377

[poseidon_sthlm]

Hello Brandon,

Thanks for your answer.

I belive I do understand your explanation but I don't agree with the conclusion. ( I don't use Set methods.)

• The situation where an overfill would occur on the daily time frame on the most liquid futures contract because of a bracket order where the distance between the SL and target is $5000 is unrealistic.
• The argument that the overfill occurs because NT doesn't know the order in which the SL and target would be filled intrabar doesn't seem to a be a consistent rule, (see for instance the bar at 8 pm in post #1). Why is that?
• How come that the overfill issue doesn't occur in the unmanaged approach, although the property ignore overfill = false?

/Poseidon_sthlm

[NinjaTrader_BrandonH]

Hello poseidon_sthlm,

Thanks for your notes.

In place of the convenience layer that the Managed approach offered, the Unmanaged approach instead offers ultimate flexibility in terms of order submission and management.

Unmanaged allows you to have full control over OCO and Overfilled orders, but removes the framework of the Managed Approach. We essentially just tell NinjaTrader to submit orders to buy and sell. SellShort and BuyToCover would be order types to enter short and close short positions, but the premise is essentially the same that we do not Enter/Exit, we just Buy/Sell.

You can read about the Unmanaged Approach following the documentation below.
https://ninjatrader.com/support/help...d_approach.htm

When using the Managed Approach, if you are not using intra-bar granularity or High Order Fill Resolution, NinjaTrader only knows if an order would have filled (between the high and low) but not the order in which the orders will have been filled as we only know the information of the finished bar. This could result in multiple orders filling on the same bar.

The best solution would be to use an unmanaged strategy that uses OCOIDs, and this would not be affected by Historical Overfills (even without intra-bar granularity or High Order Fill Resolution). Or, to add intrabar granularity to a Managed Approach strategy.

On this forum thread below you could find sample code demonstrating the Unmanaged Approach: https://forum.ninjatrader.com/forum/...13#post1163913

Here is the SampleOnOrderUpdate referece sample that you could compare with Unmanaged Template strategy on the forum thread linked above- https://ninjatrader.com/support/help...onorderupdate_ and_onexec.htm

Below is a link to another unmanaged script that demonstrates using OCO.
https://ninjatrader.com/support/foru...579#post770579