stopping and starting strategies with positions on

[NinjaTrader_Bertrand]

adamus, the issues you report in would likely be related to a database issue - until we got the expanded trace info to get closer, would you mind retrying the strategies started fresh with a new DB / SDF file?

For the OCO Id's issue: has this started only recently for you?

Were you changing any account connection options for IB in this regard?

[adamus]

I have this issue again.

IB shut down when I failed to reset the shutdown time in the TWS menu.

NT showed a pop-up asking for a shutdown and re-launch, which I did.

I am now trying to re-enable a large number of strategies. Since starting the strategies on Sunday evening at the start of the week, a couple of the strategies are still waiting to go flat before starting to trade. But half of them are running positions.

What I have decided to do, rather than flatten all positions and start again which means missing out on all these trades, is this: I'm setting NinjaTrader to "wait until flat" and with this setting, I'll re-enable all the strategies which have no positions.

Then I will change the setting to "immediately submit live working historical orders" and start up the trading strategies one by one to make sure they all start OK.

My main point is this: the configuration option "immediately submit live working historical orders" / "wait until flat" should be configurable per strategy rather than a blanket option for everything.

Thanks.

[adamus]

Just as an aside, please could you put on your list of improvements the possibility to filter the Orders tab by instrument.

I can order it by instrument, but then the orders are out of order by time.

I can order it by time, but then all the other instruments' order fill up the list.

Alternatively you could implement multiple levels of sorting, e.g. first instrument, then time.

Thanks

[NinjaTrader_Bertrand]

Thanks for those valuable suggestions adamus, we'll add them to our feedback list.

[kdoren]

I have a related question/request. When I create a New Strategy on the strategies tab, the state when it goes live is a function of when it starts, which is set by "Days to Load" in the "Time Frame" section.

I STRONGLY suggest you change this to "Start Date" so the strategy will always start/restart on the same date. Currently, if I set it to "120 days", won't it start on a different date if I disable the strategy and re-enable it a few days from now? This could cause a strategy to hold different positions and make it impossible to synchronize.

This is important to me because the strategy I'm developing trades up to 100 different instruments from a single strategy, and positions can be held several days so it's rarely flat. I'll need to have a way to disable and re-enable it without going flat. This requires that I be able to re-start the strategy on the same date every time.

BTW, in preliminary testing with an IB paper trading account, NT7 is working great with a 100-instrument strategy using 1-minute bars and IQFeed data feed. The strategy does analysis of all possible (almost 5000) pairings, with 3 1-minute data series each for a total of around 15,000 1-minute data/bool series with infinite lookback.

Bravo NT7 and kudos to the development team!

[NinjaTrader_Bertrand]

Thanks for the kind words Kevin, perhaps I'm not 100% following: if you start your strategy in 'immediately submit' mode there would be no need for it to be flat, it would recalcualte on restart and you would just need to ensure the account is then in sync to the overall strategy position shown for the instruments (the AutoSync could take care of that, too if needed via market orders).

[kdoren]

The objective is to be able to disable/re-enable a strategy, and have it always re-start in the same state with the same positions, and be in sync with the account positions.

"Days to Load" won't work right for this, because, for example, if I set "Days to Load" to 7 days, then a week later I disable/re-enable the strategy, it won't be in the same state because it will start processing bars 7 days later than it did the first time.

You really need to change this to "Start Date". Then, every time I disable/re-enable a strategy, it will remember when it started, and always start processing bars on the same date, and it when it hits real-time it will be in sync with the account positions.

[adamus]

I'm running forex strategies in an IB account and the only time I tried re-enabling the strategies with the 'sync' option, it all went to hell really badly. NT got confused by IB's "positions" where the amounts of each individual currencies are just summed up and trades and positions don't necessarily need to match. I think syncing in the NT way is suited for stocks and futures perhaps, but not forex. At least not with IB.

I had ten strategies to restart that had positions on last night, and it wasn't an experience I'd like to repeat. I had the option "immediately submit live orders" checked. From now on, I shall try to keep my systems enabled over the weekend while I have more than a couple of positions on.

(1) Of the problems I had, firstly it seems the orders that NT left open when I shut down on Friday night were not OCO. I cancelled one and expected the other to cancel, but no, it stayed open. Is the NT implementation of OCO one that is managed client-side by my instance of NT? Or is it meant to be sent as part of the orders to IB? If so, which order info column will the OCO reference appear in in IB TWS?

(2) All of the strategies tried submitting order cancellations to IB on re-enabling but made some sort of mistake. I have a dozen errors from IB like this:

"Rejected: Unable to cancel order: Internal error, reference to IB order is missing"

(3) I'd like to know whether NT is meant to cancel existing orders. The orders it submitted on re-enabling were all a bit different with their price levels for some reason presumably differences in the data from the live stream that was used initially compared to the backfilled data used to resubmit the orders at re-enabling. It would be much better to keep the original orders with the original live signals.

[NinjaTrader_Bertrand]

The objective is to be able to disable/re-enable a strategy, and have it always re-start in the same state with the same positions, and be in sync with the account positions.

"Days to Load" won't work right for this, because, for example, if I set "Days to Load" to 7 days, then a week later I disable/re-enable the strategy, it won't be in the same state because it will start processing bars 7 days later than it did the first time.

You really need to change this to "Start Date". Then, every time I disable/re-enable a strategy, it will remember when it started, and always start processing bars on the same date, and it when it hits real-time it will be in sync with the account positions.

Thanks for clarifying and your suggestions Kevin, we'll add them to our feedback list.

[NinjaTrader_Bertrand]

I'm running forex strategies in an IB account and the only time I tried re-enabling the strategies with the 'sync' option, it all went to hell really badly. NT got confused by IB's "positions" where the amounts of each individual currencies are just summed up and trades and positions don't necessarily need to match. I think syncing in the NT way is suited for stocks and futures perhaps, but not forex. At least not with IB.

I had ten strategies to restart that had positions on last night, and it wasn't an experience I'd like to repeat. I had the option "immediately submit live orders" checked. From now on, I shall try to keep my systems enabled over the weekend while I have more than a couple of positions on.

(1) Of the problems I had, firstly it seems the orders that NT left open when I shut down on Friday night were not OCO. I cancelled one and expected the other to cancel, but no, it stayed open. Is the NT implementation of OCO one that is managed client-side by my instance of NT? Or is it meant to be sent as part of the orders to IB? If so, which order info column will the OCO reference appear in in IB TWS?

(2) All of the strategies tried submitting order cancellations to IB on re-enabling but made some sort of mistake. I have a dozen errors from IB like this:

"Rejected: Unable to cancel order: Internal error, reference to IB order is missing"

(3) I'd like to know whether NT is meant to cancel existing orders. The orders it submitted on re-enabling were all a bit different with their price levels for some reason presumably differences in the data from the live stream that was used initially compared to the backfilled data used to resubmit the orders at re-enabling. It would be much better to keep the original orders with the original live signals.

adamus, if you restart strategies NT recalculates position and tries to match off what's possible, if the data for example changed which would results in different orders or positions - it would need to cancel orders and resubmit as needed unfortunately.

If entry or exit orders are cancelled or not would depend on your handling settings under Tools > Options > NinjaScript > Order Handling

For the errors you received, please forward us logs / traces for review.

[adamus]

Hi Bertrand

I'll submit the logs and traces. It was all Sunday evening around 23:00 GMT+1

Thanks
Adam

[NinjaTrader_Bertrand]

Thanks, will take a look.

[adamus]

Bertrand

I have further issues with the strategies I am running after re-enabling them last night (Sunday evening at NZ opening).

I was unable to re-enable the strategies normallly.

With the options "submit live historical orders" checked, "cancel entry orders" checked but "cancel exit orders" and "sync" unchecked, NT was unable to re-enable the strategies correctly because it generated error messages from IB (as given before) like this:

26/09/2010 23:54:54
Order
IB Simulated, Unable to cancel order:
Internal error, reference to IB order is missing affected Order:
Sell 150000 Limit @ 1.3566

This left the original target and stop loss orders live. For some strategies, one order was cancelled successfully but the other not and this resulted in further difficulties on the second attempt to re-enable, where the strategy would not submit the stop order.

I'll post more in a few minutes.

[NinjaTrader_Bertrand]

Hi Adam, I'm looking forward to any detail that you can offer - I set up a test case here over the weekend and I could not reproduce this message - are you testing with a live or paper account (normally they 'should' behave the same, but just in case...)

Thanks

[adamus]

Did you get the email regarding the overfill? I'm not sure if it's a security risk to post the link to the logs & traces zip, but I can if you don't have it still. Does NT log my license number to the trace file? That zip file has the logs and traces for yesterday from the period I was trying to re-enable everything.

So there are at least 3 issues that I'm dealing with here. One is an overfill which I'll deal with in a seperate thread.

Another is that NT can't re-enable strategies with IB with the options I choose because of this error.

NT can re-enable the strategies, but only if I follow these steps:

- I set the option "Cancel exit orders" to unchecked when I disable the strategies so that my orders remain live to protect the position

- when I re-enable, I check the option, cancel my orders, and then switch the strategies on

However I must remember to uncheck the option afterwards or I would inadvertently leave my positions open and unprotected next time I disable.

And then there is the re-enabling error where NT manages to re-use one order but fails to cancel & replace the other (same IB error), but then cannot replace the order it has re-used next time I disable and re-enable, and just seems to ignore the fact that it hasn't submitted a stop order (alarming to discover when it happens, but it is only 1 in 10 occurrence).

Lastly there is the issue with the OCO reference. Sometimes NT uses the OCO field in IB and makes the bracketed target and stop loss OCO (as intended) but sometimes it fails. For instance now all the orders that were re-entered last night have been marked OCO and this shows up in IB. However the new orders that have been entered live during normal trading since then do not have the OCO ref.

Thanks.