Is it possible to get accurate Orders Filling / Part-Filling using Market Replay?

[PrQuantumH]

Dear all,

I'm trying to detect the Part-Filled event on the TP-Order to Force exit it if it remains on Part-Filled situation after x Seconds;
By:

• Canceling it, then
• Submitting a Limit order on the nearest BID containing the remaining orders

But, using the demo data feed, I'm observing (always) a Complete Filling of the TP_Order, and not being able to detect the Part-Filling scenario.

The Question: How to get accurate (Simulated) Orders Filling/Part-Filling to study the accuracy of my code ?

Below is the simplified code and a Screenshot of the Printed events at OnOrderUpdate and OnBarUpdate. (Showing the complete filling of the TP-Orders; PartFilled and Filled are happening on the same second with total contracts value <Even while using Printing on each tick> )

private int [B]Quant [/B]= 10;
private Order [B]TP_Order [/B]= null;
private bool [B]TP_Check [/B]= false;

protected override void [B]OnStateChange[/B]()
{[INDENT]if (State == State.SetDefaults)
{[/INDENT][INDENT=2]...
Calculate = Calculate.[B]OnBarClose[/B];
...[/INDENT][INDENT]}
else if (State == State.Configure)
{[/INDENT][INDENT=2]AddDataSeries(i.FullName,BarsPeriodType.Minute, 15);   //Trading Data Series
AddDataSeries(i.FullName,BarsPeriodType.Second, 30);  //Additional Data Series for periodically 1 bar checks (Seconds)[/INDENT][INDENT]}[/INDENT]
 




  }

protected override void [B]OnBarUpdate[/B]()
{[INDENT]if (CurrentBars[1] > 50 || CurrentBars[2] > 50) return;

if(BarsInProgress == 1){[/INDENT][INDENT=2]
if( [I]LONG Conditions[/I] ) [B]EnterLong([/B]1, Quant, @"[B]Buy[/B]"[B])[/B]

if( TP Conditions ) [B]ExitLong([/B]1, Quant, "[B]TP for Buy[/B]", "[B]Buy[/B]"[B])[/B][/INDENT][INDENT]}

[B]//Checking the TP order if it still in a Part-Filled after 30 Sec to Cancel it and submit Limit order instead.[/B]
if(BarsInProgress == 2){[/INDENT][INDENT=2]
if(TP_Order!=null && (TP_Order[B].[/B]Name == "[B]TP for Buy[/B]" || TP_Order[B].[/B]Name == "[B]New BUY TP[/B]") ){

[B]//Printing TP-Order Status/Quantity @ OnBarUpdate[/B]
Print([I]"InstName: " [/I]+ InstName
+[I]" | 30 Sec has passed | BuyTP Order Status: " [/I]+ TP_Order.OrderState
+[I]" | BuyTP Order Quantity: " [/I]+ TP_Order.Quantity
+[I]" | Time: " [/I]+ Time[0]);

[B]//Checking For Part-Filled Situation[/B][/INDENT][INDENT=3]if(TP_Check){[/INDENT][INDENT=4]if(Counter == 1){[/INDENT][INDENT=5]if(TP_Order[B].[/B]OrderState == OrderState[B].[/B]PartFilled){[/INDENT][INDENT=6][B]//Get Remaining Contracts[/B]
int RemainingQuant = Quant - TP_Order[B].[/B]Quantity;

[B]//Canceling the TP Order[/B]
CancelOrder(TP_Order);

[B]//Submitting EnterShortLimit[/B]
EnterShortLimit(1, false, RemainingQuant, GetCurrentBid(1), "[B]New BUY TP[/B]");

[B]//Reseting the Check & Counter[/B]
TP_Check = false;
Counter = 0;[/INDENT][INDENT=5]}[/INDENT][INDENT=4]
}else{[/INDENT][INDENT=5]Counter++;[/INDENT][INDENT=4]}[/INDENT][INDENT=3]}[/INDENT][INDENT=2]}[/INDENT][INDENT]}[/INDENT]
 




 }

protected override void [B]OnOrderUpdate[/B](){[INDENT]
[B]//Detecting the TP order.[/B][/INDENT][INDENT]if(order.OrderAction == OrderAction.Sell && (order.Name == "[B]TP for Buy[/B]" || order.Name == "[B]New BUY TP[/B]")){[/INDENT][INDENT=2]
TP_Order = order;
TP_Check = true;

[B]//Printing TP-Order Status/Quantity @ OnOrderUpdate[/B]
Print([I]"InstName: " [/I]+ InstName
+[I]" | @OnOrderUpdate | BuyTP Order Status: " [/I]+ TP_Order.OrderState
+[I]" | BuyTP Order Quantity: " [/I]+ TP_Order.Quantity
+[I]" | Time: " [/I]+ Time[0]);

[B]//Reset the TP Order object to null if order was cancelled without any fill[/B]
if (order.OrderState == OrderState.Cancelled && order.Filled == 0)
{[/INDENT][INDENT=3]TP_Order = null;
TP_Check = false;[/INDENT][INDENT=2]}[/INDENT][INDENT]}[/INDENT]
 
  }

Thank you.

[NinjaTrader_ChelseaB]

Hello PrQuantumH,

The order.OrderState will be OrderState.PartFilled. I would recommend detecting this in OnExecutionUpdate when executions (fills) occur.

protected override void OnExecutionUpdate(Execution execution, string executionId, double price, int quantity, MarketPosition marketPosition, string orderId, DateTime time)
{
Print(execution.ToString());
if (execution.Order.OrderState == OrderState.PartFilled)
{
Print("part fill");
}
}

https://ninjatrader.com/support/helpGuides/nt8/onexecutionupdate.htm

[PrQuantumH]

Hello ChelseaB,

Thanks for your reply.

Below is the modified code, by :

• Making the detection period to be on each new Tick, and
• Adding the detection process in OnExecutionUpdate as you recommended.

==> Showing (on the log screenshot) that On all Sections (OnOrderUpdate, OnBarUpdate and OnExecutionUpdate) the Filling process happens on the same second! not allowing me to handle the PartFilled situation.

And as you see, at OnExecutionUpdate, the PartFilled situation print is happening on the same second (and with the full amount of contracts!).

private int [B]Quant [/B]= 10;
private Order [B]TP_Order [/B]= null;
private bool [B]TP_Check [/B]= false;

protected override void [B]OnStateChange[/B]()
{[INDENT]if (State == State.SetDefaults)
{[/INDENT][INDENT=2]...
Calculate = Calculate.[B]OnBarClose[/B];
...[/INDENT][INDENT]}
else if (State == State.Configure)
{[/INDENT][INDENT=2]AddDataSeries(i.FullName,BarsPeriodType.Second, 30);   //Trading Data Series
AddDataSeries(i.FullName,BarsPeriodType.Tick, 1);  //Additional Data Series for periodically 1 Tick checks [/INDENT][INDENT]}[/INDENT]
   }

protected override void [B]OnBarUpdate[/B]()
{[INDENT]if (CurrentBars[1] > 50 || CurrentBars[2] > 50) return;

if(BarsInProgress == 1){[/INDENT][INDENT=2]
if( [I]LONG Conditions[/I] ) [B]EnterLong([/B]1, Quant, @"[B]Buy[/B]"[B])[/B]

if( TP Conditions ) [B]ExitLong([/B]1, Quant, "[B]TP for Buy[/B]", "[B]Buy[/B]"[B])[/B][/INDENT][INDENT]}

[B]//Checking the TP order if it still in a Part-Filled after 30 Sec to Cancel it and submit Limit order instead.[/B]
if(BarsInProgress == 2){[/INDENT][INDENT=2]
if(TP_Order!=null && (TP_Order[B].[/B]Name == "[B]TP for Buy[/B]" || TP_Order[B].[/B]Name == "[B]New BUY TP[/B]") ){

[B]//Printing TP-Order Status/Quantity @ OnBarUpdate[/B]
Print([I]"InstName: " [/I]+ InstName
+[I]" | @OnBarUpdate | 1 Tick has passed | BuyTP Order Status: " [/I]+ TP_Order.OrderState
+[I]" | BuyTP Order Quantity: " [/I]+ TP_Order.Quantity
+[I]" | Time: " [/I]+ Time[0]);

[B]//Checking For Part-Filled Situation[/B][/INDENT][INDENT=3]if(TP_Check){[/INDENT][INDENT=4]if(Counter == 1){[/INDENT][INDENT=5]if(TP_Order[B].[/B]OrderState == OrderState[B].[/B]PartFilled){[/INDENT][INDENT=6][B]//Get Remaining Contracts[/B]
int RemainingQuant = Quant - TP_Order[B].[/B]Quantity;

[B]//Canceling the TP Order[/B]
CancelOrder(TP_Order);

[B]//Submitting EnterShortLimit[/B]
EnterShortLimit(1, false, RemainingQuant, GetCurrentBid(1), "[B]New BUY TP[/B]");

[B]//Reseting the Check & Counter[/B]
TP_Check = false;
Counter = 0;[/INDENT][INDENT=5]}[/INDENT][INDENT=4]
}else{[/INDENT][INDENT=5]Counter++;[/INDENT][INDENT=4]}[/INDENT][INDENT=3]}[/INDENT][INDENT=2]}[/INDENT][INDENT]}[/INDENT]
  }

protected override void [B]OnExecutionUpdate[/B](){[INDENT]
[B]//Detecting the [/B][B]PartFilled [/B][B]Execution.[/B]
if(TP_Order!=null && execution.Order.OrderState == OrderState.PartFilled){
[/INDENT][INDENT=2][B]//Printing TP-Order Status/Quantity @ OnExecutionUpdate[/B]
Print([I]"InstName: " [/I]+ InstName
+[I]" | @OnExecutionUpdate| BuyTP Order Status: " [/I]+ TP_Order.OrderState
+[I]" | BuyTP Order Quantity: " [/I]+ TP_Order.Quantity
+[I]" | Time: " [/I]+ Time[0]);[/INDENT][INDENT]
}[/INDENT]
 }

protected override void [B]OnOrderUpdate[/B](){[INDENT]
[B]//Detecting the TP order.[/B][/INDENT][INDENT]if(order.OrderAction == OrderAction.Sell && (order.Name == "[B]TP for Buy[/B]" || order.Name == "[B]New BUY TP[/B]")){[/INDENT][INDENT=2]
TP_Order = order;
TP_Check = true;

[B]//Printing TP-Order Status/Quantity @ OnOrderUpdate[/B]
Print([I]"InstName: " [/I]+ InstName
+[I]" | @OnOrderUpdate | BuyTP Order Status: " [/I]+ TP_Order.OrderState
+[I]" | BuyTP Order Quantity: " [/I]+ TP_Order.Quantity
+[I]" | Time: " [/I]+ Time[0]);

[B]//Reset the TP Order object to null if order was cancelled without any fill[/B]
if (order.OrderState == OrderState.Cancelled && order.Filled == 0)
{[/INDENT][INDENT=3]TP_Order = null;
TP_Check = false;[/INDENT][INDENT=2]}[/INDENT][INDENT]}[/INDENT]
   }

Question: Do you think that because I'm using a Sim-Account testing (even by using the Live-Account connection data feed), that I will not be able to get accurate Filling/PartFilling values. But I only get a simulated ones, (as you see in the SuperDOM below, the number of contracts available are very very small compared to the 50 contracts orders that I'm sending and analzing)



Thank you ChelseaB.

[NinjaTrader_ChelseaB]

Hello PrQuantumH,

Part fills typically happens extremely quickly. On a live account if an order part fills, this would trigger OnExecutionUpdate() for each part fill.

Are you asking how to force a live order to part fill? (This would not be possible as live orders are filled by matching with opposite orders)

[PrQuantumH]

Thanks ChelseaB, I truly appreciate your help.

What I want to do is :

1. [Done] Submitting a Long order (When its condition are true)
2. [Done] Submitting a TP_Order order (Take-Profit) (When its condition are true)
3. [Done] Detect the PartFilled event of the TP_Order.
4. Check after x Seconds/Ticks , if the TP_Order is still in a PartFilled situation.
5. If yes, then Cancel the TP_Order (that contains the remaining order quantity).
6. then, submitting a new TP_Order at the current BID (containing the the remaining order quantity).

In Step: (3) The issue is that it is happening at the same exact time with the Filled event, even if contracts number is very high. Only in the chart it is showing that it is taking a PartFiIled with quantity 49, then after few ticks, completing the remaining quantity 1. (Filled). But in the log (in the background) it is detecting both events at the same time with the same Total Quantity of 50.

The conclusion that I want to confirm from you, is : Is this happening because the Sim-Account is only simulating the orders filling values. It means, that it is giving a simulation experience on the chart that it is splitting the big 50 contracts quantity into a PartFIlled(49) and a Filled(+1), at the Exit (See previous chart screenshot), But in Real trading account, it should be different?

Regards,

[NinjaTrader_ChelseaB]

Hello PrQuantumH,

Trying cancel a part filled order is highly likely to cause an unable to cancel order error. A part filled order tends to keep filling very quickly. Unless the volume is extremely low, it is unlikely you would be able to get an order cancellation received by a broker before the order fills.

Simulated orders do not have a transmission time through the internet to a broker in another location and can be cancelled faster.

[CDXTrader]

Hello @PrQuantumH,

Not sure if you're still working on this but I encountered similar issues in an addon awhile back.
First thing I recommend, if you're not already doing it, use Tools > Options > Trading > Simulator > Enforce partial fills to do your testing using Simulated Data. This will slow down the partial fills enough for you to (hopefully) accurately test your logic. With a little practice you can also manipulate the trend slider to move price away from the order after partial-fill so that it stays in part filled state.

Secondly, IMHO I would stick to using the OnOrderUpdate event to check your order status because various live APIs may not give you the right Order.OrderState at the time you need it, particularly Rithmic. As recently noted in the help guide > Rithmic and Interactive Brokers Users: Due to provider API design these adapters do not guarantee the sequence of events of OnOrderUpdate, OnExecution, and OnPositionUpdate. I experienced this problem first hand and it caused very disruptive problems moving from a dev environment to a live environment.

Thirdly, although you would normally expect OnBarUpdate to fire a couple times while you're waiting for your (x)seconds to pass, it is not guaranteed and may be more accurate (depending on your use case) to start an asynchronous timer and fire the cancel order if the timer elapses before the order becomes filled. I used the System.Timers.Timer class and it seemed to work rather well and should not block the UI or any other process.

Hopefully this is helpful to you or others doing something similar. All the best.