Question: Live position synchronization

[cgeorgan]

Hello-

I thought I had a handle on exactly how NT operated with the combination of "Sync Account" and the associated parameters available (cancel all orders on disconnect, wait until flat, etc.)

In these cases, I've set my strategies to "sync account = false", and "submit live working orders immediately."

To test the behavior, I had two separate live positions on that were generated by my strategies, both of which had submitted legitimate, working GTC orders to exchanges.

I then disconnected NT and reconnected. As per the instructions, I expected the following:

• The active orders that I had submitted to the exchange that had been previously generated by the strategy to "match up" with what I had submitted prior, and be mapped to an active strategy order

What happened instead was that the local NT orders were all canceled immediately on strategy startup, and the exchange orders were subsequently deleted as well. The positions remained unchanged.

I do have "cancel entry orders when a strategy is disabled" checked, but the orders were not canceled when the strategy was disabled - they were canceled when it was re-started.

So, to you guys I ask: How exactly does NT "map" exchange orders to strategy orders, and how can I ensure that this is what's going to happen in the future?

This is important due to the need for a nightly restart while ensuring that positions and orders submitted to exchanges remain once I restart NT.

Kind Regards,
CG

[NinjaTrader_RyanM1]

Hello CG,

This page is good for breaking down possibilties depending on settings used and strategy position:

http://www.ninjatrader.com/support/helpGuides/nt7/syncing_account_positions.htm

In order for the strategy to manage orders placed with a prior instance of the strategy:
The actual order submission data for these orders must match exactly with the historical (virtual) order submission when the strategy is run. It compares date / time / price of order to look for a match.

[cgeorgan]

Hello CG,

This page is good for breaking down possibilties depending on settings used and strategy position:

http://www.ninjatrader.com/support/helpGuides/nt7/syncing_account_positions.htm

In order for the strategy to manage orders placed with a prior instance of the strategy:
The actual order submission data for these orders must match exactly with the historical (virtual) order submission when the strategy is run. It compares date / time / price of order to look for a match.

Ryan-

Thanks for the link, it's the one I used to initially determine what my behavior was.

To reiterate:

I had a live position that was entered into by a strategy. That strategy then submitted live working GTC orders for targets and exits for that position.

I then exited NT (like, 3 minutes later), restarted it, then re-started the exact same strategies. As per the description provided by NT, NT should have attempted to match those previously submitted GTC orders to the ones being held by the exchange.

Instead, NT deleted the local orders & the exchange orders, leaving the position unchanged.

[NinjaTrader_RyanM1]

Hello CG,

What happened instead was that the local NT orders were all canceled immediately on strategy startup, and the exchange orders were subsequently deleted as well. The positions remained unchanged.

I do have "cancel entry orders when a strategy is disabled" checked, but the orders were not canceled when the strategy was disabled - they were canceled when it was re-started.

As I understand it, you had exit orders working at the time of strategy disabling. How does the entry order setting relate to the exit orders scenario ?

Can you provide a reproducible scenario where entry orders are not canceled upon disabling the strategy?

[cgeorgan]

Hello CG,

As I understand it, you had exit orders working at the time of strategy disabling. How does the entry order setting relate to the exit orders scenario ?

Can you provide a reproducible scenario where entry orders are not canceled upon disabling the strategy?

I just wanted to share all relevant info.

I can reproduce this at will: All I need is for a strategy to get me into a position and issue stop/target orders, then disable the strategy and re-enable it: The strategy will cancel all the working orders.

[NinjaTrader_RyanM1]

I don't believe the cancel entry orders setting is contributing here. If you could please test with this setting checked and uncheked we can confirm this. This setting should only apply to entry orders, and not the exit orders you're having issues with.

Whether NinjaTrader can read the orders from a prior strategy instance depends on whether the execution information for the position matches between the two instances.

1) The prior instance where actual orders are placed.
2) The current instance, which will be applied to all historical data, and provide a virtual position.

It doesn't happen automatically. The date / time / price of your actual position must match the virtual position for the current, working orders to be read by the strategy.

When these match, the strategy can manage the orders.

I can reproduce what you're seeing with the orders canceled, and will verify if this is expected. It happens when the position information is distinct between an actual execution and the historical virtual execution.

[cgeorgan]

I don't believe the cancel entry orders setting is contributing here. If you could please test with this setting checked and uncheked we can confirm this. This setting should only apply to entry orders, and not the exit orders you're having issues with.

Whether NinjaTrader can read the orders from a prior strategy instance depends on whether the execution information for the position matches between the two instances.

1) The prior instance where actual orders are placed.
2) The current instance, which will be applied to all historical data, and provide a virtual position.

It doesn't happen automatically. The date / time / price of your actual position must match the virtual position for the current, working orders to be read by the strategy.

When these match, the strategy can manage the orders.

I can reproduce what you're seeing with the orders canceled, and will verify if this is expected. It happens when the position information is distinct between an actual execution and the historical virtual execution.

I think you hit on something interesting. On thing I built into my code was a "Live trading only" break, which examines the "Historical" flag when looking at historical data and won't issue new trades until it has determined that it's looking at live data.

I believe what you're suggesting to me is that my strategy needs to have run its historical trades and - if it finds itself in a "current, virtual" position, it will then search the brokerage for an *exact* replica of the trade?

On the one hand, it basically makes sense. On the other hand, I think there's a very slim chance that NT is going to have broker trades and virtual trades that match up every single time, and that's going to create a huge problem. If charts are loaded just a few seconds apart from each other, bars are formed completely differently, volatilities can vary, and a host of other issues.

There has to be some way for a strategy to merely pick up on orders submitted under that instrument...i.e. my GC strategy picks up all GC positions and orders at my broker. If this is not the case, it should really be on your programmers list, and here's why:

Holding overnight positions while allowing for a system restart. If I hold my positions overnight & expect my orders and positions to re-populate when I do my daily PC restart, then it's imperative that NT be able to pick up on these orders. Whether it's some sort of "token", "ID", or what have you, it needs to be much simpler. I programmed stuff like this myself using Trading Technology's XTAPI interface.

[NinjaTrader_RyanM1]

Yes, if your strategy is only run on live data, then that would explain your results. There is then no historical execution to compare.

The order canceling is expected in this case, documented on the sync page we've been working from:

Any active orders on the account previously generated by the strategy that does not match an active strategy order will be cancelled. Should the strategy be unable to cancel and receive confirmation on the cancellation of these orders within 40 seconds the strategy will not start and an alert will be issued.

I will forward your comments about this to development.

[cgeorgan]

Yes, if your strategy is only run on live data, then that would explain your results. There is then no historical execution to compare.

The order canceling is expected in this case, documented on the sync page we've been working from:

Any active orders on the account previously generated by the strategy that does not match an active strategy order will be cancelled. Should the strategy be unable to cancel and receive confirmation on the cancellation of these orders within 40 seconds the strategy will not start and an alert will be issued.

I will forward your comments about this to development.

Sure - but those comments are not clear. My orders were in fact generated by a live-running strategy. What I'm being told is that the orders need to be virtually created - that's a huge difference.

Thanks for forwarding this info -

[adamus]

I hope you don't mind if I jump in here with a question. It's totally related.

I'd like to know how NT knows that an order was generated by a strategy that it is starting up. In my broker account (IB TWS) I never saw any reference on the order details that an NT strategy gave.

Does NT assume that any orders on a particular instrument were generated by the strategy?

[cgeorgan]

I hope you don't mind if I jump in here with a question. It's totally related.

I'd like to know how NT knows that an order was generated by a strategy that it is starting up. In my broker account (IB TWS) I never saw any reference on the order details that an NT strategy gave.

Does NT assume that any orders on a particular instrument were generated by the strategy?

That's a very salient question, and it gets to the heart of what's going on IMHO. I get the feeling here that synchronization - the kind that's needed on restart - is not quite there yet.

I'm nervous about firing off "test cases" here myself, because everytime I test a situation where I attempt to disconnect and re-sync while trading live costs me - I hope you guys know what I mean.

Again, I'm using Trading Technologies here. Most things are fine, but I really need a reliable way to get a system restart going in order to hold overnight positions.

[adamus]

I'm nervous about firing off "test cases" here myself, because everytime I test a situation where I attempt to disconnect and re-sync while trading live costs me - I hope you guys know what I mean.

Exactly! After the hassle I had doing the restart for my strategies with multi-day positions on Sunday evening (in forex) I'm thinking that it might be worth the price of the commissions just to flatten all my account and to let the strategies sync up from scratch.

I used my IB simulation account to do the testing and it all seemed tolerable, so I've gone live with it - but I'm hoping that NT will get it working smoothly. I ended up stopping and starting each strategy multiple times with lots of faff in-between before each strategy was back up and running how it should be.

[NinjaTrader_RyanM1]

You should be able to test and verify this behavior with a sim account. There is nothing particular about this scenario that requires a live account.

In order for the orders to be read, they must match exactly.

OrderAction
LimitPrice
OrderType
Quantity
StopPrice

[adamus]

But does it assume that all orders on the strategy's instrument belong to that strategy? Or has it a method to detect only those from the strategy being enabled? And if so, what is the method?

[NinjaTrader_RyanM1]

Orders are tagged with a strategy ID, but it looks more specific than that and compares the actual execution to the historical execution when the strategy is run.

Not all orders placed with a prior strategy instance can be read by the current strategy instance - all fields of the order must align.