How can a strategy monitor ATM strategy order updates on Apex accounts?

[Felix Reichert]

Hi,

I have a strategy which uses ATM strategies for automatic trading. This is to make order execution features like multiple positions with different configurable stops and targets, and manual interventions possible. To improve the tracking (especially for rejected stops), the strategy uses AtmStrategy.StartAtmStrategy(...) instead of the methods documented here:
https://ninjatrader.com/support/helpGuides/nt8/NT%20HelpGuide%20English.html?atm_strategy_methods .htm

To keep track of the order state (knowing when an order was "finished" - filled stops and/or targets, rejection handling, etc.), the strategy uses Account.OnOrderUpdate(...). This works both for sim accounts and NinjaTrader accounts.

But when testing the strategy with an Apex Tradovate account, Account.OnOrderUpdate(...) is only called once (for the state transition to Initialized for the entry order). After that, OnOrderUpdate(...) is never called, even when manually closing the position.

So something seems differently implemented in those "external" accounts / connections. This breaks the strategy - it doesn't know when a position is flat, and the next trade can be executed.

Because the strategy uses ATM orders, the Apex / Rithmic account examples don't apply - they use the simple Enter/Exit-Long/Short(), or SubmitOrderUnmanaged(...). As documented, the Strategy.OnOrderUpdate(...) doesn't work when using ATM strategies, so I'm stuck - I really want to avoid calling GetAtmStrategyEntryOrderStatus() and then GetAtmStrategyStopTargetOrderStatus() on each OnBarUpdate() instead of the event based approach.

Is here an alternate way to keep track of the order state?

Best regards, Felix

[NinjaTrader_BrandonH]

Hello Felix,

Thanks for your post.

It is recommended to use the Atm Strategy Methods seen in the help guide link below for monitoring orders from an Atm Strategy Template.

Atm Strategy Methods: https://ninjatrader.com/support/help...gy_methods.htm
Using Atm Strategies in NinjaScript: https://ninjatrader.com/support/help...strategies.htm

Otherwise, you could consider tracking those orders from account level OrderUpdate events.

The name of the order would be the same name seen in the Orders tab of the Control Center, for example "Stop1" for a stop loss order and "Target1" for a profit target order.

OrderUpdate: https://ninjatrader.com/support/help...rderupdate.htm
Account: https://ninjatrader.com/support/help...ount_class.htm

How exactly are you defining/tracking OrderUpdate events in your script for the Atm Strategy stop/target orders?

We look forward to assisting further.

[Felix Reichert]

The linked Atm Strategy Methods have the disadvantage that you have to "poll" the order state all the time:
If I use AtmStrategyCreate(), I have to call GetAtmStrategyEntryOrderStatus() and GetAtmStrategyStopTargetOrderStatus() in each OnBarUpdate(), since they are not events.
Before I rewrite all of the order management logic, I would like to know if the polling of the order state causes any traffic to the market place, or runs only locally, and if there is a more efficient way than calling GetAtmStrategyStopTargetOrderStatus() on each OnBarUpdate()? Also in previous testing I had the impression that the data returned by GetAtmStrategyStopTargetOrderStatus() is wrong if one of the stops was rejected by the market place.
The inadequate handling of rejected stops in NinjaTrader (at least for ATM orders) is the main reason for the complex order "house-keeping" in the strategy. I will probably create another post for that.

Like I wrote in my initial post, I already use the account level OrderUpdate events, but they only fire once for each order (when entry order is Initialized), if I'm not using the sim account.
So NinjaTrader behaves completely different for sim vs. "real" accounts. After testing with other accounts (for example the NinjaTrader brokerage CQG account), I could reproduce the same problem (not further calls to account level OrderUpdate).

The event handler for Account.OrderUpdate() is added in OnStateChange() when State == State.DataLoaded.
For each call, the current OrderUpdate() event handler updates a structure containing suborder data (filled quantity and order state) for entry / stops / targets, and this data is correlated by using e.Order.GetOwnerStrategy(), and the orderId of the entry order, which the strategy created itself using Account.CreateOrder(). Like mentioned, this Account.OrderUpdate() is only called once, so the strategy thinks that the order is not even submitted yet. Because the strategy permits managing multiple orders of multiple signals in parallel (automatic, "half-automatic" and manual), correlating the order updates to the created ones is essential.

[NinjaTrader_BrandonH]

Hello Felix,

Thanks for your notes.

For a NinjaScript that uses Atm Strategy Methods the recommended way to monitor orders would be to use GetAtmStrategyEntryOrderStatus() and GetAtmStrategyStopTargetOrderStatus().

These methods would run locally on your machine and from my understanding would not cause "traffic" to the market place.

Executions from ATM Strategies will not have an impact on the hosting NinjaScript strategy position and PnL - the NinjaScript strategy hands off the execution aspects to the ATM, thus no monitoring via the regular NinjaScript strategy methods will take place (also applies to strategy performance tracking). This information could be found in the Using Atm Strategies help guide page linked in post # 2.

That said, Account level OrderUpdate events could be subscribed to to track orders submitted using Atm Strategy Methods.

The first stop order submitted with an ATM Strategy will have an Order.Name of "Stop1". The second stop order submitted with an ATM Strategy will have an Order.Name of "Stop2", and so on. The first target order submitted with an ATM Strategy will have an Order.Name of "Target1". The second target order submitted with an ATM Strategy will have an Order.Name of "Target2" and so on.

See the help guide documentation below for more information.
Account: https://ninjatrader.com/support/help...ount_class.htm
OrderUpdate: https://ninjatrader.com/support/help...rderupdate.htm

While undocumented/unsupported, GetOwnerStrategy() can be used to retrieve an ID. Since this is an undocumented/unsupported method, we cannot provide support for using GetOwnerStrategy() in a NinjaScript or any unexpected behavior that may occur.

Below is a forum thread you could view for more information on this topic and sample code:

https://forum.ninjatrader.com/forum/ninjatrader-8/platform-technical-support-aa/1085887-strategyid-needed?p=1085921#post1085921

You may consider viewing this script from the Ecosystem Use App Share which can listen to OnOrderUpdate and use GetOwnerStrategy to identify an order as being owned by an Atm strategy:

https://ninjatraderecosystem.com/user-app-share-download/atmstrategyidentifier/

[Felix Reichert]

Hi Brandon,

Thanks for your fast answer - it's good to know that repeatedly calling GetAtmStrategyStopTargetOrderStatus() doesn't have a negative impact on the connection to the market place. If I can't get Account.OrderUpdate events firing reliably, I will try this approach, this is however a bigger change which I would like to avoid.

I know that ATM strategies are executed "outside" of the strategy scope, and therefore my strategy relies on on Account level OrderUpdate events instead of Strategy.OnOrderUpdate(). However this event is only fired once when NOT using the sim account, which is my problem.

Is there any setting or similar which fixes this? How can I diagnose what the problem is?
When putting a breakpoint in Visual Studio (or just printing info to the output) in the event handler, I can verify that it is only called once, so it's not a problem of correlating IDs, or calling GetOwnerStrategy() incorrectly. I will try the ATMStrategyIdentifier example tomorrow with a non-sim account to see if it works or not - the approach is very similar to my order management code.

Best regards, Felix

[NinjaTrader_BrandonH]

Hello Felix,

Thanks for your notes.

If you print out the e.Order object in OnOrderUpdate when subscribed to Account OrderUpdate events, do you see prints appearing in the NinjaScript Output window stating the order information for the object?

Keep in mind that when using Rithmic, the sequence of events (OnOrderUpdate, OnExecutionUpdate, OnPositionUpdate) are not guaranteed due to provider API design.

Please create a simple reduced test script that demonstrates reproducing the behavior in question regarding the Account.OrderUpdate events along with the exact steps and settings you are using to reproduce the behavior using the simple test script so we may investigate this matter further.

Note that a reduced copy refers to a copy of the script that contains the minimum amount of code needed to reproduce the issue. All other code is commented out or removed.

To create a copy of your script to modify, open a New > NinjaScript Editor, select your script, right-click in the Editor, select 'Save as', name the script, and click OK.

To export the script go to Tools > Export > NinjaScript AddOn.

We will then use the reduced test script and the steps you provide to try and reproduce the behavior on our end and investigate further.

[Felix Reichert]

When printing the order info in the Account.OrderUpdate handler, only one line is generated when using an Apex account (when the entry order is Initialized).
I will create a minimal sample so that you can reproduce the problem, thanks for your assistance!

[Felix Reichert]

Nevermind, I found the problem: To be able associating the order update events with known newly created orders, the strategy used the OrderId of the entry order. For some strange reason, this OrderId changes when the account is not the Sim101 account, or the connection is not playback. I now have a check whether the strategy runs with a "real" account / connection, or for testing, and use either the entryOrder.OrderId, or the AtmStrategy.Id as key to find the known orders. When the IDs didn't match, the rest of the handler wasn't executed => it seemingly wasn't called.

I would have liked to always use the AtmStrategy.Id, but in playback, the creation of the ATM strategy only returns the created ATM strategy instance after all stops and targets are submitted - if some of them are rejected, the order update handler would not be able to detect this and react.

Some sample output with an Apex account showing that the entry OrderId changed from 57bb45c1fa214a519102cd348487eaec to 51607770668 :

OnOrderUpdate() Time: 19:42:21.227 OrderName: Entry OrderID: 57bb45c1fa214a519102cd348487eaec Order.ID: 974 OrderState: Initialized Filled: 0 ATM strategy ID: (null) OrderData NOT found
OnOrderUpdate() Time: 19:42:21.227 OrderName: Entry OrderID: 57bb45c1fa214a519102cd348487eaec Order.ID: 974 OrderState: Initialized Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
ATM order created, entryOrder.OrderId: 57bb45c1fa214a519102cd348487eaec entryOrder.Id: 974ATM strategy ID: 326056079
OnOrderUpdate() Time: 19:42:21.324 OrderName: Entry OrderID: 51607770668 Order.ID: 974 OrderState: Submitted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.326 OrderName: Entry OrderID: 51607770668 Order.ID: 974 OrderState: Accepted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.326 OrderName: Entry OrderID: 51607770668 Order.ID: 974 OrderState: Working Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.325 OrderName: Entry OrderID: 51607770668 Order.ID: 974 OrderState: Filled Filled: 1 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.456 OrderName: Stop1 OrderID: 3196700a08c74298b87902b03bdef055 Order.ID: 975 OrderState: Initialized Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.464 OrderName: Target1 OrderID: 00884a04f8604150a30afdf8f5da8476 Order.ID: 976 OrderState: Initialized Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.531 OrderName: Stop1 OrderID: 51607770674 Order.ID: 975 OrderState: Submitted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.533 OrderName: Stop1 OrderID: 51607770674 Order.ID: 975 OrderState: Accepted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.533 OrderName: Stop1 OrderID: 51607770674 Order.ID: 975 OrderState: Working Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.551 OrderName: Target1 OrderID: 51607770677 Order.ID: 976 OrderState: Submitted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.553 OrderName: Target1 OrderID: 51607770677 Order.ID: 976 OrderState: Accepted Filled: 0 ATM strategy ID: 326056079 OrderData NOT found
OnOrderUpdate() Time: 19:42:21.553 OrderName: Target1 OrderID: 51607770677 Order.ID: 976 OrderState: Working Filled: 0 ATM strategy ID: 326056079 OrderData NOT found

[NinjaTrader_BrandonH]

Hello Felix,

Thanks for your notes.

The orderID generated by NinjaTrader is only used temporarily as a placeholder until the actual order id for the order as it gets created on the brokerage servers is returned.

Here is a forum thread discussing this topic: https://forum.ninjatrader.com/forum/...atrader-broker

Yes, the stops and targets from an Atm Strategy would need to both be submitted as you have noted.

You would need to make sure proper logic is used in the script so that stops and targets are submitted on the correct side of the market and are not being rejected.

If you choose to modify the script to submit orders using the Managed Approach or Unmanaged Approach you could track orders in OnOrderUpdate() / OnExecutionUpdate() by their signal names instead of by an orderID and have more control over the order handling/management. You could also use RealtimeErrorHandling and OnOrderUpdate() to determine the behavior of a strategy when the strategy places an order that is returned "Rejected".

Managed Approach: https://ninjatrader.com/support/help...d_approach.htm
Unmanaged Approach: https://ninjatrader.com/support/help...d_approach.htm
RealtimeErrorHandling — https://ninjatrader.com/es/support/h...orhandling.htm

[Felix Reichert]

Hi Brandon,

Thank you for your further tips, I will consider changing to the unmanaged approach. What really slows down the development are the many subtle differences between playback / sim101 and real account behavior - not only the placeholder IDs (which is new and unexpected to me thanks for the clarification!), but also some strange things in playback, like entry orders being Partially Filled, then Cancelled by the strategy, and after that unexpectedly Filled. I would expect that an order in terminal state (Cancelled) can't change it's state anymore.

Is there a way to test the unmanaged approach with more realistic behavior than the playback connection and / or the sim101 account, but without paying for each transaction? The funded accounts like Apex will also cost money if too many trades go into the wrong direction (which is normal when testing and debugging).

Best regards, Felix

[NinjaTrader_ChelseaB]

Hello Felix,

You are using the addon approach to send orders with atm strategies attached using AtmStrategy.StartAtmStrategy() in this script, is this correct?

https://ninjatrader.com/support/helpGuides/nt8/startatmstrategy.htm

This example demonstrates using the addon approach to send atm orders. This approach could also be used in an indicator and handling all behavior through the account updates. If you are using the addon approach with the account, no need for this to be a strategy at all.

https://forum.ninjatrader.com/forum/ninjatrader-8/platform-technical-support-aa/93107-saving-atmstrategy-templates#post794486

Below is a link to an indicator in the User App Share that gets information about atm strategy orders you may also find very helpful.

https://ninjatraderecosystem.com/user-app-share-download/atmstrategyidentifier/

"but also some strange things in playback, like entry orders being Partially Filled, then Cancelled by the strategy, and after that unexpectedly Filled. I would expect that an order in terminal state (Cancelled) can't change it's state anymore."

I'm understanding you have event handlers added to the <Account>.OrderUpdate / <Account>.ExecutionUpdate events, is this correct?

https://ninjatrader.com/support/helpGuides/nt8/orderupdate.htm

https://ninjatrader.com/support/helpGuides/nt8/executionupdate.htm

From the <Account>.OrderUpdate event, are you printing the e.Order.ToString() at the top of the event handler method (before any conditions) and getting information about each order?
Same with the e.Execution.ToString() with the <Account>.ExecutionUpdate event handler?

If so, can you provide the output from this so that we can help analyze what is going on?

[Felix Reichert]

Hi Brandon,

AtmStrategy.StartAtmStrategy() - yes, I use the Addons approach, and it works fine. Also, after changing the ID for orders from the OrderId of the entry order to the strategy.Id (but just for "real" accounts - not for playback or sim101), my order management works fine, thanks!

Unfortunately, I don't have the logs anymore for the partially filled orders in playback, but I can create a new forum topic as soon as I can reproduce it.
From what I remember, the sequence is:

1. Entry.OrderState == PartFilled
2. Stop1.OrderState == Rejected
3. (strategy cancels all non-terminal suborders, including the entry order, and "sells" all positions from the partial entry order. e.g. 2 of 3)
4. Entry.OrderState == CancelPending
5. Entry.OrderState == Cancelled
6. Entry.OrderState == Filled

That last fill means that now there is an imbalance (account not flat anymore), since the strategy sold less positions in step 3 than the whole filled entry order. I also had the impression (not verified) that this doesn't happen in all cases, so a workaround just for the playback connection gets more complicated.

Which leads back to my question: Is there a way to test (without wasting money) with more realistic behavior than the playback connection? Like having a "market place simulator" running locally?
The documentation even mentions that the partial fills are sometimes inserted to force developers to make their strategies more robust. But I think that a real market place would not fill an entry which it cancelled before.

Best regards, Felix

[NinjaTrader_ChelseaB]

Hello Felix,

You can test on the Sim101 account built-in to NinjaTrader, but if you want the account to behave like your broker, you will need to contact your broker and inquire if you can get a demo account to test on.