Working order getting cancelled outside of the strategy

[scouratier]

Hey friends!

[EDIT: Partial Fills don't seem to be relevant. I'm seeing the behavior with no Partial Fills as well]

Context: I'm trying to understand why my working orders are getting cancelled. Based on my investigation, this is NinjaTrader deciding to cancel the orders, not anything I'm doing in the strategy, I think this has to do with some knowledge I'm missing on how NT is handling the orders behind the scenes.

Some Pseudo code:
- in OnBarUpdate, I enter a long for 10 contracts
- in OnExecutionUpdate, I create my StopLoss and 2 TakeProfit targets. The first TP has 4 orders, the second TP has 6 order.
- time goes by
- I'm lucky! my first TakeProfit is hit as a partial fill
- in OnOrderUpdate, I move my StopLoss using ChangeOrder()
- in the Logs, I see Cancel requests for my StopLoss and for my other TakeProfit
- in the Logs, I see the order be cancelled
- I'm left with 6 working orders and no TP and no SL (so.. very very bad).

Attached is a screenshot of the Logs with some compelling artwork.

More in depth:
- I do in fact have code paths that can cancel orders. This is where I focused most of my time.
- in OnOrderUpdate, if StopLoss is hit, I cancel all the order and clean up
- in OnOrderUpdate, if all of the TakeProfits are "Filled" and I have 0 filled orders, I cancel all orders and clean up
- I've added logging to both these scenarios
- I've also added logging to the cancel methods themselves.
- When the scenario happens (partial fill -> cancel all orders), I no trace of my cancel methods in the logs.
- As part of my logs, I'm also dump the state of all my orders (main entry, take profits and stoploss). They are look normal before the cancel.
- I've also added debug level logs that log out every single OrderUpdate and ExecutionUpdate, and I just added every PositionUpdate.
- For my managed approach, I use:
-- EnterLong / EntryShort to start my order
-- ExitLongLimit / ExitShortLimit for my TakeProfits
-- ExitLongStopMarket / ExitShortStopMarket for my StopLoss
- This all works fine in the Strategy Analyzer.
- I've been testing this on Sim accounts (as you can see in the screenshot above).

What I've done so far:
- I worked on a lot on my OnOrderUpdate logic to make sure I wasn't calling the cancels on accident.
- My theory yesterday had to do with OCO order. This led me down 2 paths:
--> I did have signalnames on my stoploss and TakeProfits. I removed those
--> I switched to unmanaged
- Both the above approaches led to the same result. Canceled orders on partial fills.'
[EDIT: Not true. As it turned out, switching to unmanaged does in fact fix the behavior]
- Added lots of logging..

So that's where I'm at right now. From what I can tell, NT makes a decision to cancel my orders. I don't understand why it would do that, and if there's something I can do to stop it.

Cheers!

Stephen

[NinjaTrader_Jesse]

Hello scouratier,

From the given details I couldn't really say what may be happening, have you tried to disable any logic that you have which cancels orders to confirm that you are not in fact cancelling the orders?

You can also try using TraceOrders to confirm that the internal handling rules are not part of the reason.

Beyond that we would very likely need a simplified sample of what logic was used during that situation to see why that may be happening. I would suggest trying to use one of the samples from the help guide that uses OnOrderUpdate/OnExecutionUpdate to manage targets along with partial fills to see if you get a similar outcome or not. There is a sample in the following page you can use which handles partial fills: https://ninjatrader.com/support/help...ub=onexecution

[scouratier]

Hi Jesse,

Nice to meet you!

1) Have I tried disabling the cancel logic: yes! I was able to isolate the cancel logic and repro the issue.

2) TraceOrders! This sounds promising. I'll google it, but if you have docs handy, that would help

3) I'm working on getting logs of healthy and unhealthy runs (it's not easy, because I need to wait for a trade to trigger AND it has to hit TP1 AND the it has to have partial fills)
I'll take a look at the sample tho! If nothing else, it will either confirm I'm doing it the right way, or teach me something.

Thanks!

S.

[NinjaTrader_Jesse]

Hello scouratier,

You can find details about trace orders in the help guide:

https://ninjatrader.com/support/helpGuides/nt8/traceorders2.htm?zoom_highlightsub=traceorders

[scouratier]

Thanks! (also, I'm actually very familiar with this sample. It was my starting point).

Cheers

[scouratier]

Hello again!

After taking a few days to review my code, look into tracing and try a few different things, here are some of the things I've learned:

- I'll need to update my original post. Switching to Unmanaged does in fact fix the behavior. The second stop loss is not canceled and I'm not left with an orphan entry.
- I'd like to shy away from using Unmanaged for now tho, because there are plenty of things I don't want to / don't know how to manage yet.

Here is a trace log of event that lead to an orphan entry:

https://gist.github.com/scouratier/1...99a87539312e56 (had to use Gist.. went over the 10000 character limit).

The line that stand out to me is (line 47):

2023-10-23 10:06:23:049 Cbi.Position.RemoveFirstEntry: cancel order: orderId='3adec34ce099421e887aabff951881e6' account='Sim101' name='static1 10/23/2023 10:06:00 AM' orderState=Working instrument='MNQ 12-23' orderAction=Sell orderType='Limit' limitPrice=14689.75 stopPrice=0 quantity=5 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=602 time='2023-10-23 10:06:00' gtd='2099-12-01' statementDate='2023-10-23'

As a reminder / context:
- I get into a Long for 10 contracts
- In OnExcutionUpdate, I create a StopLoss (10 contracts) and 2 TakeProfits (5 contacts each)
- TP0 (it's called static0) is hit (for 5 contracts)
- I try to move StopLoss and change quantity to 5 contracts
- TP1 (static1) gets canceled by ... something
- StopLoss gets canceled by .. something
- I'm left with an entry with 5 contracts, no stop loss or take profit

This seems to happen only with Partial Fills (but I need to get logs to confirm that).

Stephen

[scouratier]

For comparison, this is the healthy run:

https://gist.github.com/scouratier/7b9ccdd429495a35012c1a21ea109d30

Same flow (order, SL, TP0, TP1, Hit TP0, then eventually hit SL).

[NinjaTrader_Jesse]

Hello scouratier,

If using unmanaged works I would likely suggest taking that route. As previously mentioned we would need a specific sample to know if the logic you are using is expected to have that happen or not.

Its possible this could relate to the strategies stop target handling setting as that has some specific scenarios with partial fills that are mentioned. https://ninjatrader.com/support/help...sub=stoptarget

[scouratier]

And here is the sample I've been working on:

https://gist.github.com/scouratier/d...c15277391fc6bb for code that's formatted.

OnBarUpdate(){
if (goLong){
if (entry.order == null)
{
if (IsUnmanaged)
{
entry.order = SubmitOrderUnmanaged(0, OrderAction.Buy, OrderType.Market, sizing, 0, 0, null, "Long " + today);
}
else
{
entry.order = EnterLong(0, sizing, "Long " + today);
}
}
}
if (goShort){
if (entry.order == null)
{
if (IsUnmanaged)
{
entry.order = SubmitOrderUnmanaged(0, OrderAction.Sell, OrderType.Market, sizing, 0, 0, null, "Short " + today);
}
else
{
entry.order = EnterShort(0, sizing, "Short " + today);
}
}
}
}
OnExecutionUpdate(){
if (execution.Order.Name.StartsWith("Long") || execution.Order.Name.StartsWith("Short"))
{
if (execution.Order.OrderState == OrderState.Filled)
{
if (inOrder.IsLong)
{
try
{
if (IsUnmanaged)
{
entry.stopLoss.order = SubmitOrderUnmanaged(0, OrderAction.SellShort, OrderType.StopMarket, inOrder.Filled, 0, sl, null, "Stop " + inOrder.Name);
}
else
{
entry.stopLoss.order = ExitLongStopMarket(0, true, inOrder.Filled, sl, "Stop " + inOrder.Name, null);
}
if (entry.stopLoss.order == null)
{
Log.Error("Unable to create a long SL " + entry.stopLoss.order);
}
}
catch (Exception e)
{

Log.Error("Exception while trying to create a long SL " + e);
}
}
if (inOrder.IsShort)
{
try
{
if (NS.IsUnmanaged)
{
entry.stopLoss.order = SubmitOrderUnmanaged(0, OrderAction.BuyToCover, OrderType.StopMarket, inOrder.Filled, 0, sl, null, "Stop " + inOrder.Name);
}
else
{
entry.stopLoss.order = ExitShortStopMarket(0, true, inOrder.Filled, sl, "Stop " + inOrder.Name, null);
}
if (entry.stopLoss.order == null)
{
Log.Error("Unable to create a short SL " + entry.stopLoss.order);
}
}
catch (Exception e)
{
Log.Error("Exception while trying to create a short SL " + e);
}
}
if (inOrder.IsLong)
{
if (NS.IsUnmanaged)
{
tempTP.order = NS.SubmitOrderUnmanaged(0,
OrderAction.SellShort,
OrderType.Limit,
_config.breakoutTradingSettings.takeProfits.target s[i].sizing,
tempTarget,
0,
null,
_config.breakoutTradingSettings.takeProfits.style. ToString() + i + " " + inOrder.Time);
}
else
{
tempTP.order = NS.ExitLongLimit(0,
true,
_config.breakoutTradingSettings.takeProfits.target s[i].sizing,
tempTarget,
_config.breakoutTradingSettings.takeProfits.style. ToString() + i + " " + inOrder.Time,
null);
}
}
for (int i = 0; i < config.targets.Count(); i++)
{
tempTarget = config.targets[i].amount;
if (inOrder.IsLong)
{
if (IsUnmanaged)
{
tempTP.order = SubmitOrderUnmanaged(0,
OrderAction.SellShort,
OrderType.Limit,
config.targets[i].sizing,
tempTarget,
0,
null,
"static" + i + " " + inOrder.Time);
}
else
{
tempTP.order = ExitLongLimit(0,
true,
config.targets[i].sizing,
tempTarget,
"static" + i + " " + inOrder.Time,
null);
}
}
if (inOrder.IsShort)
{
if (IsUnmanaged)
{
tempTP.order = SubmitOrderUnmanaged(0,
OrderAction.BuyToCover,
OrderType.Limit,
config.targets[i].sizing,
tempTarget,
0,
null,
"static" + i + " " + inOrder.Time);
}
else
{
tempTP.order = NS.ExitShortLimit(0,
true,
config.targets[i].sizing,
tempTarget,
"static"+i + " " + inOrder.Time,
null
);
}
}
entry.takeProfits.targets.Add(tempTP);
}
}
}
}
OnOrderUpdate()
{
if (orderState == OrderState.Filled)
{
if (order.Name.StartsWith("Long") || order.Name.StartsWith("Short"))
{
entry.order = order;
entriesTaken++;
}
if (order.Name == "Exit on session close")
{
Log.Debug("State of order before Cancel (Exit on session close): " + entry.ToString());
KillEntry();
KillTakeProfits();
KillStopLoss();
}
if (order == entry.stopLoss.order)
{
Log.Debug("State of order before Cancel (stopLoss): " + entry.ToString());
KillEntry();
KillTakeProfits();
KillStopLoss();
}
for (int i = 0; i < entry.takeProfits.targets.Count(); i++)
{
if (order == entry.takeProfits.targets[i].order)
{
Log.Debug("State orders: " + entry.ToString());
// entry.currentFill return the total number of orders that are not in Filled state
if (entry.currentFill() <= 0)
{
Log.Debug("State of order before Cancel (static1or0): " + entry.ToString());
KillEntry();
KillTakeProfits();
KillStopLoss();
}
else
{
if (breakEven)
{
double price = entry.order.AverageFillPrice;
if (entry.order.IsLong)
{
price = entry.order.AverageFillPrice + breakEvenCalculated.points;
}
if (entry.order.IsShort)
{
price = entry.order.AverageFillPrice - breakEvenCalculated.points;
}
Log.Debug("Moving StopLoss: " +price+" on "+entry.currentFill()+" orders");
// move stoploss does a checks to make sure it's safe to move stoploss
// and then calls changeorder()
MoveStopLoss(price, entry.currentFill());
}
else
{
Log.Debug("Changing Order quantity for StopLoss: " + entry.currentFill());
UpdateQuantityStopLoss(entry.currentFill());
}
}
}
}
}
}

[scouratier]

Happy Monday Jesse!

I'll explore those breadcrumbs. The documentation is .. not as insightful as I'd like. I'm currently on StopTargetHandling.PerEntryExecution. I'll go away for a bit and try do some experiments.

Stephen

[NinjaTrader_Jesse]

Hello scouratier,

Unfortunately that sample is not complete and is outside of what I could test. If you are having a problem with the managed approach I would suggest making a new empty strategy and extract only the code needed to see that problem. Make sure the code runs one time in realtime to demonstrate the problem. Omit any extra code like the unmanaged code along with any custom methods, make the script as simple as possible so that it highlights only the problematic code. Please include the full script as a .cs file attachment to the forum.

[bfalls]

scouratier,

Did you ever get an answer to the managed order solution. I am having a similar issue and curious if you solved this problem. Here is my post: https://forum.ninjatrader.com/forum/...ously-canceled
Thanks for any help or insights.

[scouratier]

Oh my goodness. You are making me revisit this nightmare! =).

I never found a reliable solve to my problem, either in managed or un-managed orders. My underlying idea and strategy changed since then, and I don't need to place orders the way I used to (TLDR, I exit a trade a partial TP using a market order). My best theory had to do with OCO. My idea to solve it was to move from 1 entry, 2 stop losses and 1 stop loss, to move to 2 entries, 2 TPS and 2 SL. But reading your post, that's more or less what you are doing already, so .. it probably would not have worked.

Sorry =(.

Stephen

[bfalls]

Solved my problem, see this post. https://forum.ninjatrader.com/forum/...65#post1309065