error switching position in high-frequency bars - RC1

[DaveE]

Using NT7 RC1:
If NT tries to switch position (from Long to Short or vice versa) in a short duration bar (say 5 seconds), at least while trading on IB, then ALL Stop Loss and Target orders can be cancelled, thus leaving the new position unprotected.
I have even seen cases where the new position is double the size requested in the strategy.
I have also seen a case where NT closed the strategy (having detected some conflict condition), but did not close the unprotected position.

Steps to reproduce the problem:
1) Create a simple strategy called DontRunThisLive (no parameters) as below.

2) Run the strategy on a 5 second FX data series, against IB papertrade account.

3) Monitor the Orders in NT (& IB Portfolio tab).
You may find the majority of position changes work fine, but it will not take long before you see the following in IB:
Four bracket orders are visible at the same time, then all get cancelled. I assume this has something to do with the full set of actions not completing within the bar period. So if the new bracket orders dont become active before the five second bar completes, then they are being cancelled by something (NT on the start of new bar)?

This appears to indicate something defective in NT error handling in cases where bar completes before order changes requested in the bar have become active.

DontRunThisLive code:
This is simply designed to switch frequently between Long and Short positions, in theory always protected by Stop Loss and Target orders.

protected override void Initialize()
{
SetProfitTarget("S1", CalculationMode.Ticks, 15);
SetStopLoss("S1", CalculationMode.Ticks, 20, false);

SetProfitTarget("L1", CalculationMode.Ticks, 15);
SetStopLoss("L1", CalculationMode.Ticks, 20, false);

DefaultQuantity=100000;
TraceOrders=true;
CalculateOnBarClose = true;
}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
if (CurrentBar<10) return;

// Condition set 1
if (Close[0] > Open[0]
&& Low[0] > Low[1])
{
Print(string.Format("Should EnterLong with DefaultQuantity={0}, EntriesPerDirection={1}, EntryHandling={2}",DefaultQuantity,EntriesPerDirec tion,EntryHandling ));
EnterLong(DefaultQuantity, "L1");
}

// Condition set 2
if (Close[0] < Open[0]
&& High[0] < High[1])
{
Print(string.Format("Should EnterShort with DefaultQuantity={0}, EntriesPerDirection={1}, EntryHandling={2}",DefaultQuantity,EntriesPerDirec tion,EntryHandling));
EnterShort(DefaultQuantity, "S1");
}
}

[NinjaTrader_Brett]

Hello,

Are you running this on CalculateOnBarClose = True or False?

Also, Can you please upload yor log file form the day you ran this and this occured.

My Documents--> NinjaTrader 7--> Log--> log.20101105

Also let me know what time this occured.

I look forward to assisting you further.

[DaveE]

Hello Brett,

Calculate On Bar Close=true (strategy settings).
I will email the log, but this should not be needed since NT can reproduce the error, by following the steps I provided. It should not take more than about 15 minutes before you get an occurrance of the error.

[NinjaTrader_Austin]

DaveE, I will have someone get back to you on Monday as I do not have access to an IB paper trading account.

[NinjaTrader_Brett]

DavE,

It would be better if I had the logs from your accout in this case. So that I know what specifically to look for. I have run this strategy before but most likely what you are experiencing here is in flight executions.

http://www.ninjatrader.com/support/helpGuides/nt7/risks_of_electronic_trading_wi.htm

Thanks for sending in when you can.

[DaveE]

In flight executions

Hello Brett,

I have sent NT Log & trace and TWS log info.

The definition of In Flight Executions here http://www.ninjatrader.com/support/h...trading_wi.htm
implies that it is a problem that only affects NT requests to close a position.
But I think that page should probably really say it also affects requests to Reverse a position. For instance the previous position has been closed and its brackets are still in the process of being cancelled while NT is requesting setting up of the reverse position.
For all I know NT may handle this correctly for other brokers, but maybe it has problems with handling errors from IB native OCO handling?
I have experienced a similar problem trading manually using the Super DOM. When using REV (Reverse) I have found the following occasional anomalies:

• New reversed position is twice the size of previous opposite position.
• or new reversed position is unprotected (even though an ATM strategy was active that specified a stop loss)

So I am coming to the conclusion that reversing a position using NT is not a safe operation when trading with IB. Of course if a problem occurs after hitting REV in Super DOM you can correct the situation manually.
But these problems are not acceptable in a strategy. The problem as I see it is that NT does not seem to be waiting until brackets for the previous position have been fully de-activated (not just requested to be cancelled), before starting to set up the reverse position.
As a work around (until NT fixes the problem) I am going to stop attempting to use NT "reverse" functionality, and instead wait for a minimum fixed number of seconds (not sure how long this should be) after the previous position has been closed, before attempting to set up a new position.

[NinjaTrader_Brett]

Hello,

I responded to your email. Best is to only use one form of communications so that we do not cross paths in our communication.

However what I outlined in your email is that your using EnterLong() and EnterShort(). Take the EnterShort() out of the equation and the strategy runs flawlessly. Add it back in and we hve the In Flight Executions.

Heres the reason.

On some bars EnterShort() and EnterLong() are being called at the same time with your entry conditions.

Also EnterShort() when called after an EnterLong() will reverse your position.

Whats occuring is the say you are long, you call and EnterShort() and a Profit Target get hit at the same time. This is an in flight execution.

Along with if you hit the reverse button.A reverse is a QTY of 2 to reverse your position. 1 to go flat and 1 to go short. However if a stop loss or profit target gets hit this same second and you actually have a flat position by the time the REV goes into effect with its short 2 order this leaves you with a short 2 position.

Hopefully this makes since to you. Let me know if you need further explanation on the above.

To avoid this you would need to have a delay or be carefully not to have 2 exit conditions that could trigger at the same time to cause an in flight execution. Some coding styles are more prone to this as others.

[DaveE]

Hello Brett,
Re:

On some bars EnterShort() and EnterLong() are being called at the same time with your entry conditions

This is not true.
This stress-test strategy:

• Will only call EnterLong() if the bar just closed was "green".
• Will only call EnterShort() if the bar just closed was "red".

Since its not possible for the bar just closed to be both green and red (Close both higher and lower than Open), its not possible these are both being called in the same bar.

It is of course possible for this strategy to call EnterLong() on one bar then EnterShort() on the next (or vice versa).
It is of course by design that this strategy is likely to be Long (from a previous bar request) at the time when EnterShort() is called, or Short (from a previous bar request) at the time when EnterLong() is called. This is no different to the sample strategies such as SampleMACrossOver where its possible in a "chop" phase for a fast MA to cross above a slower MA one bar, but below again the next. So if you add SetStopLoss and SetProfitTarget to SampleMACrossOver you will find exactly the same problem (it just may take longer to come up).

Your message does not explain why the new reverse position has its bracket orders cancelled (so is unprotected).

Please could you answer the following questions:
1) Is NT supposed to be able to detect that that a previous position, including its bracket orders, has been deactivated at IB, when these were set up using SetStopLoss and SetProfitTarget?

2) If the answer to 1 is Yes, then isn't this a bug: NT attempting to open a reverse position before detection of this above state is confirmed?

3) If the answer to 1 is No, then is there an alternative method (such as use of ExitLongStop), under Managed Orders, for setting up Stop Loss & Target, where NT does correctly wait for this detection (before attempting to set up a new order)?

4) Is it the case that a check for MarketPosition can give status Flat between the position being closed, but before the associated brackets have been deactivated at IB?

[NinjaTrader_Brett]

Hello,

I did not delve into the exact definition of your If statements for your longs and shorts however you could be correct. In either reguard and EnterShort() is called at the same time that a profit target or stop is executed thus causing this behaivior.


You would be able to program in your own OverFill and Order handling to add in any In Flight execution code you would need to tell the strategy what to do when this occurs.

3)Yes, if you are reversing position in the confines of a profit target and stop you will need to code for this since this could cause an in flight execution.

Please see the sample code below on how to do this, this isnt a 1 to 1 exact what you need however you will be able to modify this to fit your needs. Essentially you need to cancel the stop loss/profit target before you submit a reversal order and get confirmation that they were filled before submitting the reversal order.

http://www.ninjatrader.com/support/forum/showthread.php?t=7499

Let me know if I can be of further assistance.

[DaveE]

Hello Brett,
Thanks for your reply.
I will check Josh's sample, and look into switching to the unmanaged approach.

[NinjaTrader_Brett]

Hello,

Sounds good. You can also do this in a unmanaged aproach however you need to take some action before entering short.

Such as with our ATM Strategies when you hit the close button on the superDOM and you hve a profit target and stop loss active. What we actually do is modify the profit target bloew the market price to fill instantly.

Once this is done you could then Reverse the position.

This order of order entry will prevent this from occuring as well. Essentially you just need to manage by filling or cancelling the current targets and stops.

Let me know if I can be of further assistance.