Question about recovering trade from connection loss

[markdshark]

A strategy I'm running writes a logfile with the current position which order details such as entry price, name, entry times, side, etc. and looks like this:

5,S-REG3-5,REG3,09:27:07,Feb,7,Tuesday,9,short,75.26,0,75.9 8,75.25,837,RECEND

When the strategy starts up it if PositionAccount.MarketPosition != MarketPosition.Flat it checks the recovery file and reads these details into an order object and pushes it onto a list. I would then like it to continue monitoring price movements and exit this position at the appropriate time based on moving stops.

Note I've also tried jumping into recovery logic with Positions[tradedInstrument].MarketPosition != MarketPosition.Flat and it goes there which I understand to mean it recognizes these positions were created by the strategy to begin with.

I also have the following set:

IsAdoptAccountPositionAware = true;
StartBehavior = StartBehavior.AdoptAccountPosition;

However I get this:

Reading Trade Record...
Position recovered...
2/7/2023 10:09:01 AM Strategy 'A1Dev/302631936': Entered internal SubmitOrderManaged() method at 2/7/2023 10:09:01 AM: BarsInProgress=1 Action=BuyToCover OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='S-REG3-5-close' FromEntrySignal='S-REG3-5'
2/7/2023 10:09:01 AM Strategy 'A1Dev/302631936': Ignored SubmitOrderManaged() method at 2/7/2023 10:09:01 AM: BarsInProgress=1 Action=BuyToCover OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='S-REG3-5-close' FromEntrySignal='S-REG3-5' Reason='SignalName does not have a matching FromEntrySignal to exit'

Is there no way to read in an order signal name and all and slide back into Managed exits?

Mark

[NinjaTrader_BrandonH]

Hello markdshark,

Thanks for your post.

It sounds like you may need to add GetRealtimeOrder() into your script so your strategy will have an order object to reference for the entry order. There are a couple of places in the help guide that help to explain how this works:

• https://ninjatrader.com/support/help...storicaltolive
• https://ninjatrader.com/support/help...ltimeorder.htm

Additionally, the SampleOnOrderUpdate strategy is an example that also utilizes GetRealtimeOrder(). It calls GetRealtimeOrder() in OnStateChange() when State == State.Realtime to transition any historical order object references to match the new real-time order that is submitted live to the account. That sample may be found here:
https://ninjatrader.com/support/help...and_onexec.htm

When using Adopt Account Position, you likely need to add logic to catch if you are enabling the strategy during an open long or short position. The example at the link below demonstrates how to add conditions that check once (using a bool) for an open long or short position when enabling the strategy. If there is a position, exit orders are submitted without a "fromEntrySignal" value:

• https://forum.ninjatrader.com/forum/...06#post1169606

[markdshark]

Thanks Brandon.

For simplicity, I am only dealing with adopting existing positions and not old orders.

So I'm still slightly confused cos it's not a backtest "order" that's transitioning. It's a position that was placed by the strategy before it was disabled for whatever reason. When it's reenabled I'd like to pick up that position and continue as though nothing happened.

Currently I am writing a custom record containing only specific pertinent details of the order and then reading that file back into an "orderClass" object which looks like this:

private class orderClass {
public Order order;
public int tradeNumber, timesAe, timesFe, paraStopBar, initialHeading;
public string Name, tradeWeekDay, tradeMonth, tradeDate, tradeHour;
public string longShort, closeCondition, regime;
public double mfe, mae, firstFe, firstAe, high, low, accFactor;
public double targetPrice, stopLoss, xp, entryPrice, exitPrice;
public bool tp1Hit, tp2Hit, tp3Hit, tp4Hit, gainsLock = false;
public bool tpHit, paraStopBE, paraStopSet, accelerated, orderInProgress, removed;
public bool feMinCheckOk, aeMinCheckOk;
public DateTime entryTime, exitTime, mfeTime, maeTime;
}; // orderClass



/// recover trades from disconnect
if (firstRun && UseRecoveryFile && PositionAccount.MarketPosition != MarketPosition.Flat)
recoverPosition();


private void recoverPosition(){

if(
Positions[tradedInstrument].MarketPosition != MarketPosition.Flat )
readRecords();

}

private void readRecords()
{

Print("Reading Trade Record...");

string line =File.ReadAllText(recoveryFile);
string[] record = line.Split(new char[] {','}, StringSplitOptions.None);
int splitCounter = 0;
o = new orderClass();
ordList = new List <orderClass>();

foreach (string field in record)
{

splitCounter++;
if (splitCounter == 1)
o.tradeNumber = int.Parse(field);
if (splitCounter == 2)
o.Name = field;
if (splitCounter == 3)
o.regime = field;
if (splitCounter == 4)
o.entryTime = DateTime.ParseExact(field, "HH:mm:ss", null);
if (splitCounter == 5)
o.tradeMonth = field;
if (splitCounter == 6)
o.tradeDate = field;
if (splitCounter == 7)
o.tradeWeekDay = field;
if (splitCounter == 8)
o.tradeHour = field;
if (splitCounter == 9)
o.longShort = field;
if (splitCounter == 10)
o.entryPrice = double.Parse(field);
if (splitCounter == 11)
o.targetPrice = double.Parse(field);
if (splitCounter == 12)
o.stopLoss = double.Parse(field);
if (splitCounter == 13)
o.xp = double.Parse(field);
if (splitCounter == 14)
o.paraStopBar = int.Parse(field);

}

ordList.Add(o);

Print("Position recovered...");
o.orderInProgress=false;

} // readRecords


So at the end of startup I have a custom orderClass object filled with details about the existing account position, but I'm realizing that's not the same as having an Order object. But then I'm thinking that should be ok since I simply proceed with the price based logic in OnMarketUpdate and operate on my custom class to have the strategy place and move soft stops or take profit. Or should I be doing something like Serializing the Order object for recovery and then reading that back in and doing a call to GetRealTimeOrder on that instead? But then the OrderState would be Filled already cos it's actually a position not an "order" and according to the docs that means it would return null.

And critically... the crux of the matter is when the strategy later places an order to enter an opposite position long/short will this existing position be taken care of as a managed order? I believe I understood from your last reply if I wanted to close it manually myself say at a take profit level I could simply submit a managed order command without a "fromLabel" like so:

ExitLong(tradedInstrument, 1, closeLabel, "");

Mark

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

Once an order is placed by a strategy and then the strategy is disabled, once the strategy is reenabled the order that was placed would be considered a historical order to the reenabled strategy instance.

Please try adding GetRealtimeOrder() in your script so your strategy will have an order object to reference for the entry order. See the help guide documentation in post # 2 for more information about using GetRealtimeOrder().

Further, you likely need to add logic to catch if you are enabling the strategy during an open long or short position since you are using Adopt Account Position. Please see the forum thread for a reference sample that demonstrates the use of Adopt Account Position.

[markdshark]

Hi Brandon... But according to the docs if an order status is filled.... In other words the order is now an open position, then GetRealTimeOrder will return null. See screenshot below. So what's the point of using GetRealTimeOrder? Seems like it doesn't apply here.



Plus as I have shown above I have already added logic to catch if the strategy is being enabled. To quote from above:


/// recover trades from disconnect
if (firstRun && UseRecoveryFile && PositionAccount.MarketPosition != MarketPosition.Flat)

recoverPosition();

private void recoverPosition()
{

if(Positions[tradedInstrument].MarketPosition != MarketPosition.Flat )

readRecords();

}


Perhaps if I rephrase the question:

1. I am enabling a strategy with an open position. No orders exist. Just an open position. Let's say it's a long position in this example.
2. The strategy reads in order details from a recovery file so it knows entryPrice, stoploss, etc.
3. Now let's say I would like the strategy to close the order.

QUESTION 1: Can I simply submit a managed order as follows:

ExitLong(tradedInstrument, 1, closeLabel, "");

4. Perhaps instead I would like the strategy to go short.

QUESTION 2: If the strategy issues a managed short entry as shown below, will it exit this old position and enter a short position?

EnterShort(tradedInstrument,1, "MyShortOrder");


The IsAdoptAwareExample uses unmanaged orders. I am trying to stick with managed orders for now and I am only interested in exiting the position as per internal strategy logic, or reversing the existing position with a single managed Enter order.


Many thanks.
Mark

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

If the strategy calculates that you are in a Long position, you could call ExitLong() to exit the long position. For example, if the strategy calculates that you are in a Long position when the strategy is enabled and your condition to call ExitLong(); becomes true, the Exit order would be placed to exit the long position.

If you would like the strategy to go short then yes, you could call the EnterShort() method to have the strategy go from a Long position to a short position.

This is because Entry() methods will reverse the position automatically. For example, if you are in a 1 contract long position and call EnterShort() -> you will see 2 executions, one to close the prior long position and the other to get you into the desired 1 contract short position.

Managed Approach: https://ninjatrader.com/support/help...d_approach.htm

[markdshark]

Thanks Brandon. That's great news. So to summarize:

1. I can have the strategy enter a position and write out the account position details to a text file.
2. I can set IsAdoptAccountPositionAware = true; in State.SetDefaults.
3. I can disable or disconnect and re-enable the strategy and read in the recovery file.
4. I can then simply call Managed Approach entries and exits such as:

ExitLong(tradedInstrument, 1, closeLabel, "");
EnterShort(tradedInstrument,1, "MyShortOrder");

... And my strategy will merrily exit or reverse my position as required.

This all sounds great. It does beg the question though:

What exactly does "IsAdoptAccountPositionAware = true;" do anyway? It does appear I could do steps 1, 3 & 4 while omitting step 2 and regardless of what the strategy's start behavior was set to.


Best,
Mark

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

When a strategy is enabled, it processes historical data to determine trades that the strategy would have made on the data that is already on the PC/chart and to determine what position the strategy is in.

The Start Behavior option Adopt Account Position would be used if you want the strategy to inherit the Account Position upon enablement. This requires additional programming such as setting IsAdoptAccountPositionAware = true.

Note that Strategy positions are separate from actual Account positions.

For example, if you would like your strategy to disregard the historical virtual Strategy Position that it calculated and have it start in the same position as the real-world Account Position, the Adopt Account Position Start Behavior could be used.

Typically, the Immediately Submit Start Behavior option is used to have a strategy resume a position after disabling/enabling. Immediately Submit automatically submits working orders from when the strategy processed historical data, and assumes the strategy position and account position are where you want it when you enable the strategy. If the strategy already had live orders running, the orders will resume with the new enablement of the strategy if they match the historically calculated orders. If the orders calculated from historical data do not match the live working orders, the live working orders will be cancelled and replaced by those calculated from historical data.

See this help guide page for more information about how Adopt Account Position works and information about the Immediately Submit Start Behavior: https://ninjatrader.com/support/help..._positions.htm

See this help guide page for more information about Strategy Position vs Account Position: https://ninjatrader.com/support/help..._account_p.htm

[markdshark]

Ok I think I understand the bit about ignoring historical calculated trades and starting from the current Account Position. As a final thought could one simply recreate this same behavior by adding the line if(State == State.Historical) return; and then enabling the strategy with a behavior like "immediately submit" and first checking if an account position exists and reading in the recovery file and adopting that. Same end result right?

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

Adding if (State == State.Historical) return; to the top of your strategy logic means that historical processing is skipped. This means the strategy will not calculate a position from processing historical data.

The strategy will then always start from a flat position because it has not calculated any orders.

To have the strategy resume a position after disabling/enabling, you could just set the Start Behavior to Immediately Submit and not include the if (State == State.Historical) return; code in the script.

[markdshark]

Sorry.. Just trying to understand.

Does not adding if (State == State.Historical) return; and setting the Start Behavior to Immediately Submit become equivalent to setting IsAdoptAccountPositionAware = true; given both situations would disregard the historical virtual Strategy Position and also require additional logic to assume the existing position?

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

Say you are in a 1 Long position. If you skip historical processing and use Immediately Submit, the Account Position would be 1 Long and the Strategy Position would be Flat. Since historical processing is skipped in the strategy, the strategy position would always be flat when the strategy is enabled. Strategy positions are separate from actual Account positions.

Then if you were to call EnterShort() to change direction, the 1 Long Account Position would be flat and the Strategy Position would be 1 Short. Meaning the Strategy Position and Account Position would be out of sync.

Adopt Account Position would be used if you want the strategy to inherit the Account Position upon enablement. Meaning the Strategy Position would inherit the Account Position.

You could find more information about Immediately Submit and Adopt Account Position on the help guide documentation linked in post # 8 and linked below.

https://ninjatrader.com/support/helpGuides/nt8/syncing_account_positions.htm

[markdshark]

Hi Brandon thank you for that explanation and thank you also for your patient replies while I struggle to get a grasp on this.

Here's my confusion. My strategy has code that informs it if it's being started in "recovery-aware" mode or not. If it's started in "recovery aware" mode it checks the Account position and if there is one it looks for a log file with position details. If it finds one it reads these details into a custom orderClass object one variable of which is an Order object. During regular operations that variable will hold an Order object but in recovery mode there is no order object, just position details like entryPrice, stopLoss, target, etc.

The strategy then adds the custom orderClass object to a list. It knows for example not to go long again because it knows that it's already long on the basis of the orderClass object it read in. It then proceeds to iterate through the list at every marketUpdate. It updates the recovery record if stops move and also checks whether the position should be closed. If so it issues a managed Exit order with no "from-signal" and closes the position. On the other hand if it encounters an opposite signal it issues a managed Enter order which closes the previous position and opens a new one. The new open order has a signal name and the strategy uses "from-signal" names to manage the position further.

This appears to be working:

Enabling NinjaScript strategy 'A1Dev/303400422' : On starting a real-time strategy - StartBehavior=AdoptAccountPosition AccountPosition=MCL 09-23 1S EntryHandling=All entries EntriesPerDirection=1 StopTargetHandling=Per entry execution ErrorHandling=Stop strategy, cancel orders, close positions ExitOnSessionClose=True / triggering 1800 seconds before close SetOrderQuantityBy=Strategy ConnectionLossHandling=Recalculate DisconnectDelaySeconds=10 CancelEntriesOnStrategyDisable=True CancelExitsOnStrategyDisable=False Calculate=On each tick IsUnmanaged=False MaxRestarts=4 in 5 minutes

Reading Trade Record...
Position recovered...

1,RECOV-RECOV-S-REG3-1,REG3,10:26:13,Aug,14,Monday,10,short,82.1,0,83.1 ,82.05,0,RECEND

Swing found at Bar: 9
SwingStop for RECOV-RECOV-RECOV-S-REG3-1 : 83.2
Stop unacceptable. Changed to Max stop: 83.1
ParaStop for order: RECOV-RECOV-RECOV-S-REG3-1 : 83.1

8/14/2023 11:50:34 AM Strategy 'A1Dev/303400422': Entered internal SubmitOrderManaged() method at 8/14/2023 11:50:34 AM: BarsInProgress=2 Action=Buy OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='L-REG3-1' FromEntrySignal=''

08/14/2023 11:50:34 AM - Closing RECOV-RECOV-RECOV-S-REG3-1 on LONG condition at 82.61

Net: -51 Cumulative Profit: -51

# 1. - LONG -
------------------
Entry for L-REG3-1 : 82.61
ATR: 38
Swing found at Bar: 16
SwingStop for L-REG3-1 : 81.76
ParaStop for order: L-REG3-1 : 81.76

...shows it closed a recovered short position after I stopped and started the thing 3 times due to other issues. The original run left a record with a position name "S-REG3-1". Subsequent updates rewrote the record adding the prefix "RECOV-" to the name each time (minor bug will tackle later)

So basically whatever mode I start the program in, I never want it to consider history. Just the present situation and of course it can run back along the BarsArray and use historical data for calculations.

So I'm confused about what IsAdoptAccountPositionAware brings to the table. With your example above using ImmediatelySubmit, if I was in a 1 Long position and skipped historical processing, the Account Position would be 1 Long and the Strategy Position would be Flat during startup but the first thing the strategy would do is realize it was in recovery mode and immediately get back in sync. Then if I were to EnterShort the managed approach would enter 2 orders, one to close the long position and another to open a new Short position so the strategy would never be out of sync.

Of course this only works as long as the strategy opened the original long position as it would then write the recovery record OR I could doctor one by hand but the net effect would be the same.

My thoughts are that the above is basically a re-implementation of IsAdoptAccountPositionAware but it requires that recovery logfile and that using IsAdoptAccountPositionAware would somehow allow me to skip the tapdancing above and simply adopt the account position but I seem to be struggling to make that final connection. Maybe I'm only imagining there is one.

I've examined SampleOnOrderUpdate. That's basically just going from historical to realtime using GetRealTimeOrder

I examined IsAdoptAwareExample and that just wraps an existing position with a 50 tick profit target and stoploss using unmanaged orders

I examined AdoptAccountPositionAndSubmitProtectiveSLPT orders and that basically does the same as above but with managed orders.


Best,
Mark

[NinjaTrader_BrandonH]

Hello Mark,

Thanks for your notes.

If you are programming your strategy in the manner that you mentioned to "realize it was in recovery mode and immediately get back in sync" then you could consider skipping historical processing and use Immediately Submit. This is something that you will need to test on your end though to ensure the strategy behaves as you are expecting it to behave.

Setting IsAdoptAccountPositionAware to true allows you to use the Adopt Account Position start behavior which is generally used to inherit the Account Position upon enablement.

When using Adopt Account Position any active orders on the account previously generated by the strategy that does not match* an active strategy order will be cancelled. Should the strategy be unable to cancel and receive confirmation on the cancellation of these orders within 40 seconds the strategy will not start and an alert will be issued.

Matching active orders on the account will then be mapped to the active strategy orders. Any remaining active strategy orders that cannot be successfully paired will be submitted live and the strategy will then try to sync your Account Position to your Strategy Position. Only one strategy with this setting can be started at a time for an individual account and instrument.

The account and instrument the strategy is started on must not have any working orders which were submitted outside of the strategy, or by another instance of the same strategy. If an order is detected, the strategy can not be started until these orders have been manually managed.

You could try testing your strategy when skipping historical processing and using Immediately Submit and test your strategy using Adopt Account Position to see how the strategy behaves for each start behavior and decide which start behavior suits your overall goals the best.