Does Order.OrderId stay persistent?

[bltdavid]

According to this post from the NT8 beta period, the Order.OrderId never changes
and can be used like NT7's old Order.Token.

[That is, the OrderId is permanent and can be used to track orders, such as when one
needs to track orders using the Account.OrderUpdate event handler, such as might be
done in a trade copier. A permanent artifact inside each Order object that is guaranteed
to never change is very important.]

Ok, but ...
Why does the documentation for Order say the OrderId value can change?

Is the documentation correct?
Under what circumstances does the value of OrderId change?

[jeronymite]

bltdavid No doubt NT Support will provide a comprehensive response. Nevertheless, my own recent experience shows that an initial OrderId is generated when an Order is created (e.g. a6066c013f644f0d907d05d0f4ef20d6). That remains unchanged until the Order is at the broker/exchange and the first update to the Order is sent back to NT8. At that point, the broker/exchange has generated a new OrderId (e.g. 141356312) and that replaces the original one. I discovered this specific behaviour when tracking OrderState in OnOrderUpdate and saw that the same OrderState was being repeated in consecutive updates of the same Order, which seemed strange, and on comparing the full Order attributes between such apparently identical updates, realised it is only the OrderId changing in the manner mentioned.

Ideally, as you highlight, one would also like to have a unique, permanent "OrderIdentifier" that is generated at Order creation and never changes. (Separately, I'd also like to see the Order object have a Tag, like various WPF elements ... but that may be an ask too far.)

Thanks.

[bltdavid]

That remains unchanged until the Order is at the broker/exchange and the first update to the Order is sent back to NT8. At that point, the broker/exchange has generated a new OrderId (e.g. 141356312) and that replaces the original one.

Thanks for your insights.

Which broker was that?
How many brokers did you test with?

It's possible NT support will wash their hands and state the OrderId is
under the control of the broker technology being employed.

But is that really the case?

Let's have some fun with some wild speculation.

[EDIT: Recall the docs say OrderId is "a string representing the broker
issued order id value" -- thus my concern for more info on how & why
the OrderId could change.]

Let's say with CQG/Rithmic the OrderId may stay permanent, but
then, with TDA or IB, or with Crypto, those brokers may update it,
say due to the way they handle 'change order' requests -- who knows,
these are just my wild examples of the possibilities involved when NT
says something is 'under control of the broker' and not NinjaTrader.

I'm not sure if "a string representing the broker issued order id value"
means "string under total control of the broker, and therefore could
change at any time (or not) on a per broker basis".

But, perhaps I'm being too pessimistic.

How so?
The documentation implies the OrderId can change due to transitioning
from a Historical order to a Realtime order ... but no other circumstances
causing a change to the OrderId are discussed.

If that's the only circumstance, I'm in the clear, because I subscribe
to Account.OrderUpdate events only after State.DataLoaded has
been reached (thus, I don't care about H -> R order transitions,
that is, I wish to ignore historical orders, I only intend to handle
new realtime 'live' orders submitted after the H -> R transition.)

Thus, I think I'm safe, but am I?
(safe because of NT's comment in that very old beta period post,
which I'm not sure can be trusted, due to contradictions in the docs.)

Like you, I'd like more clarity from NT Support.

[jeronymite]

Clarity is always welcome. The broker for the example I gave is FXCM, using a demo account. Also, it is what I observe in an AddOn, and so is always real-time. My tests have been limited to this context (because of the nature of the overall testing I'm currently conducting). [Apologies for extensive data following, but it gives a complete view worth considering.]

Here's another example full sequence of changes with update times and OrderIds, where the submitted Stop Order was subsequently cancelled very soon after. These are trace statements from within OnOrderUpdate:

13:39:59.226 1bd2fd3d9a3947f19fce3f773e63b867 Initialized
13:40:08.240 1bd2fd3d9a3947f19fce3f773e63b867 Submitted
13:40:09.317 1bd2fd3d9a3947f19fce3f773e63b867 CancelPending
13:40:09.369 141356999 CancelPending
13:40:09.369 141356999 Accepted
13:40:09.369 141356999 Working
13:40:09.450 141356999 CancelSubmitted
13:40:13.598 141356999 Cancelled​

This is what is in the NinjaTrader trace file for the same sequence, edited to remove irrelevant items from the messages:

13:39:59:226 (DEMO NT FXCM 5) Cbi.Account.CreateOrder: orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Initialized id=53687 time='13:39:59' id=-1
13:39:59:226 (DEMO NT FXCM 5) Cbi.Account.Submit0: realOrderState=Initialized isPendingSubmit=False orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Initialized id=53687 time='13:39:59'
13:39:59:226 (DEMO NT FXCM 5) Cbi.Account.Submit1: realOrderState=Initialized orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Initialized id=53687 time='13:39:59'
13:40:08:209 (DEMO NT FXCM 5) Fxcm.Adapter.Submit1: orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Initialized id=53687 time='13:39:59'
13:40:08:209 (DEMO NT FXCM 5) Fxcm.Adapter.ToFxcmOrderValueMap: 'orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Initialized id=53687 time='13:39:59' change=False
13:40:08:210 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=Submitted orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=Submitted time='13:40:08'
13:40:08:752 (DEMO NT FXCM 5) Fxcm.Adapter.OnOrderRow: updateAction='Insert' orderID='141356999' status='W' tradeID='74235281' offerID=8 stage=O type='SE' buySell='B' primaryId='' requestTxt='1bd2fd3d9a3947f19fce3f773e63b867|SE|GTC|2|0|0.63474|2|' requestId='CFDDEMO01_E589117A50D05B4EE053E12B3C0AC4D3_12012022032849210165_-553'
13:40:09:317 (DEMO NT FXCM 5) Cbi.Account.Cancel0: realOrderState=Submitted orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=CancelPending id=53687 time='13:40:09'
13:40:09:317 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=CancelPending orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=CancelPending time='13:40:09'
13:40:09:317 (DEMO NT FXCM 5) Cbi.Account.QueueCancel: realOrderState=Submitted orderId='1bd2fd3d9a3947f19fce3f773e63b867' orderState=CancelPending id=53687 time='13:40:09'  <<<<<<<
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=Accepted oldOrderId='1bd2fd3d9a3947f19fce3f773e63b867' orderId='141356999' orderState=Accepted time='13:40:08'  <<<<<<<
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback.RetryCancel: orderId='141356999' orderState=Working id=53687 time='13:40:08'
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.Cancel0: realOrderState=Accepted orderId='141356999' orderState=Working id=53687 time='13:40:08'
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=CancelPending orderId='141356999' orderState=CancelPending time='13:40:09'
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.Cancel1: realOrderState=CancelPending orderId='141356999' orderState=Working id=53687 time='13:40:08'
13:40:09:369 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=Working orderId='141356999' orderState=Working time='13:40:08'
13:40:09:449 (DEMO NT FXCM 5) Fxcm.Adapter.Cancel1: orderId='141356999' orderState=Working id=53687 time='13:40:08'
13:40:09:449 (DEMO NT FXCM 5) Cbi.Account.OrderUpdateCallback: realOrderState=CancelSubmitted orderId='141356999' orderState=CancelSubmitted time='13:40:09'
13:40:11:796 (DEMO NT FXCM 5) Fxcm.Adapter.OnOrderRow: updateAction='Update' orderID='141356999' status='S' tradeID='74235281' offerID=8 stage=O type='SE' primaryId='' requestTxt='1bd2fd3d9a3947f19fce3f773e63b867|SE|GTC|2|0|0.63474|2|' requestId='CFDDEMO01_E589117A50D05B4EE053E12B3C0AC4D3_12012022032849210165_-696'
13:40:11:797 (DEMO NT FXCM 5) Fxcm.Adapter.OnOrderRow: updateAction='Delete' orderID='141356999' status='C' tradeID='74235281' offerID=8 stage=O type='SE' primaryId='' requestTxt='1bd2fd3d9a3947f19fce3f773e63b867|SE|GTC|2|0|0.63474|2|' requestId='CFDDEMO01_E589117A50D05B4EE053E12B3C0AC4D3_12012022032849210165_-696'​

And for completeness, the NinjaTrader log file, with irrelevant items in the messages removed:

13:40:08:240|1|32|Order='1bd2fd3d9a3947f19fce3f773e63b867/1001671555' New state='Submitted'
13:40:09:369|1|32|Order='141356999/1001671555' New state='Accepted'
13:40:09:370|1|32|Order='141356999/1001671555' New state='Working'
13:40:09:449|1|32|Order='141356999/1001671555' New state='Cancel submitted'
13:40:13:598|1|32|Order='141356999/1001671555' New state='Cancelled'

It's a somewhat complex dance under the covers, as evidenced by the trace file. Nevertheless, it would seem to indicate that in real-time, from the moment of Order creation until the broker actually provides a response, the OrderId is as created by NinjaTrader (it can't be anything else until it reaches the broker for the first time anyway), and then morphs to a genuinely broker-generated OrderId. This is clear in the trace file as marked (<<<<<<< at 09:317 to 09:369), where the original OrderId changes between two sequential events to become the oldOrderId and the (presumably) broker-supplied OrderId becomes the current OrderId. Interestingly, it seems that various NinjaTrader internal variables track a variety of "Ids".

So, are you "safe" in real-time from a change to an OrderId? Not if this is representative of things; and even if this is an outlier and only for this specific broker, it still shows a change in real-time is possible.

Additional clarity from NT Support will be welcome.

Thanks.

[NinjaTrader_Jesse]

Hello bltdavid,

You would not want to use the orders ID for tracking purposes in NinjaScript, you can instead use the orders name or store the order to a variable if you need to keep track of its values during its lifetime. The order ID is generated by the broker once the order has a state so tracking an order from the very beginning to end by its id would not be possible in NinjaScript. The order id could be used for manual tracking purposes if you needed to compare your broker statement to NinjaTrader logs.

You can find a sample of how to track orders from NinjaScript in the following link: https://ninjatrader.com/support/help...and_onexec.htm

[bltdavid]

What if the order name is empty?

Try entering an order from Chart Trader using ATM Strategy set to 'None'.

In that specific case, there is no order name.

[bltdavid]

Hmm, the Order object has a field called 'id', which is an integer number.

This field is printed when using Order.ToString().

It looks like this field is just a sequential number, incremented for each
new order when it's created.

It doesn't appear to ever change -- is this the permanent and unique
'token' inside each Order that I've been looking for?

Jesse, what is Order.id and how is it used?
Is Order.id permanent -- does it ever change?

jeronymite, what does your testing reveal regarding the 'id' field?

[NinjaTrader_Jesse]

Hello bltdavid,

You can find the documented property descriptions for an Order object in the following link, ID is not documented but OrderID is which is the brokers returned id. Its not suggested to try and track orders by id numbers, you should use an order Object as a variable if you wanted to keep track of an order. https://ninjatrader.com/support/help...lightsub=order

To track a manual order you would have to look at what type of order it was and its other values to know if its of interest and then save it to a variable if you wanted to refer back to it later in a convenient way. For any other order such as a strategy order that has a name you can just use its name to track it because the name will stay the same.

[jeronymite]

bltdavid I haven't looked closely at it, actually. A quick look at the complete order lifecycle of various Orders in the trace file seems to indicate it is constant, but not all entries include it, so it's not definitive. I suppose one would need to explicitly track it on every OnOrderUpdate, as I have done previously for OrderState. If that were to indicate constancy, one might presume it to be so ... but the inevitable thought is "not supported, could/might change". And even that monitoring approach is not comprehensive, as the trace file shows entries that are not aligned to any callback that can be monitored in code. It's really a case of needing a purpose-created, documented, unique identifier within the Order object ... which is where we started, basically.

Thanks.

[bltdavid]

I haven't looked closely at it, actually. A quick look at the complete order lifecycle of various Orders in the trace file seems to indicate it is constant, but not all entries include it, so it's not definitive. I suppose one would need to explicitly track it on every OnOrderUpdate, as I have done previously for OrderState. If that were to indicate constancy, one might presume it to be so ... but the inevitable thought is "not supported, could/might change". And even that monitoring approach is not comprehensive, as the trace file shows entries that are not aligned to any callback that can be monitored in code. It's really a case of needing a purpose-created, documented, unique identifier within the Order object ... which is where we started, basically.

Thanks, my friend.

My permanent unique identifier for each Order object is now,

string MyToken = o.Name + o.Id.ToString();

and later when I call o.CreateOrder() the name argument is set
to MyToken ... this means my OrderUpdate event handler can
later receive an order on the leader account, look through the
list of Orders of each follower acct, testing for,

if (o.Name == MyToken)
   // found follower order replicated from leader acct

The leader acct only needs to replicate submit, change, and cancel orders
to each follower acct -- this is (presently) the best unique identifier I can
think of. With change and cancel orders on the leader acct, we must
rummage through Orders of each follower acct to find the existing
order previously replicated via the initial leader submit replication.

Anyways,
Using Sim101, I had previously found that OrderId was constant, but I
have now tested with Rithmic and can confirm there is no persistence,
the OrderId changes itself just like you described -- so thank you for
the heads up on that.



In summary, I switched to using "o.Name + o.Id" ... which so far seems
to work perfectly as a unique identifier.

[jeronymite]

Good approach bltdavid. Should work well.

My own approach is to use a naming convention for all Orders that incorporates 8-digit random integers along these lines:

• Txxxxxxxx --> A collection of Orders conceptually treated as a single "trade"
• Txxxxxxxx/Nyyyyyyyy --> An individual entry Order within the trade
• Txxxxxxxx/Nyyyyyyyy/Xzzzzzzzz --> An individual exit Order for the corresponding entry Order within the trade

Because my AddOn software provides the ability to have a single action that can execute multiple Orders (grid) on multiple Instruments (basket) in multiple Accounts (multi-Account) all within a single construct I call a "trade", my event logging to disk always associates the Order name with the relevant Account and Instrument to explicitly identify the position affected.

This works well because my software is treated as the source of all Orders in play within all positions arising from the trade. The Orders themselves can be changed/cancelled external to my software (e.g. ChartTrader) and the unique names make all that obvious and the changes are handled appropriately; and where Orders have names that are NT8-generated (e.g. Close), I associate the Order with the relevant position and treat it accordingly.

If "new" Orders that are "unknown" to my software are created/executed externally to my software (e.g any NT8 Order submission mechanism, and even broker-initiated "Orders" for things like margin calls), the software detects and handles all those accordingly as well, adjusting to changes that affect known positions as appropriate.

It's a complex dance; but then, as you yourself know well also, nobody said this trading stuff was simple.

Wishing you good trades and positive PnLs.

Thanks.