Overfill triggers market order doubling position size

[adamus]

I have already submitted an email from NT using the send facility to attach the logs and traces.

Here is the zip file: TracesAndLogs.zip

This is probably linked to the difficulties I had re-enabling the strategies last night.

I am running strategies on several forex pairs and at the moment I am running them side-by-side in my IB live and IB simulation accounts.

On one strategy NT registered an overfill and immediately submitted a market order which it seems was completely unrequired. I had to reverse the order. Fortunately this was on the simulation account.

The strategy on the live account did not suffer and overfill. It made the same execution on a stop order.

What I don't understand is whether the second order - the market order - was the problem or an attempt by NT to correct the overfill.

I had to buy back to correct the situation and now the Positions tab for the DU account shows rubbish values - it has really got totally out of sync with the actual positions.

[sju10544]

Hi adamus,

had the same issue (see http://www.ninjatrader.com/support/f...ad.php?t=33870).
The team NT examines just the problem...
Hope this helps.

[adamus]

It's done it again.

Here are the traces & logs: TracesAndLogs.zip

It occurred in both the live account and the simulation account and lost me $50 before I could close the overfill.

[adamus]

I'm just checking up on the strategy to work out whether it did something wrong.

I can't see anything wrong.

The market order is called "Close" so it's not from the strategy, it's generated by NT. Why did it do that? There was only one order near the market - the other part of the bracket was miles away. Here's the logging I did:

2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 BarUpdate Short O[0.84965] H[0.85060] L[0.84960] C[0.85060]
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 BarUpdate OOs: BuyToCover [email protected] STP (Accepted), BuyToCover [email protected] LMT (Working)
**NT** An over fill was detected on order 'Order='512731311/DU58298' Name='Exit1Stop' State=Filled Instrument='$EURGBP' Action=BuyToCover Limit price=0 Stop price=0.8509 Quantity=0.1M Strategy='ThreeDucks002' Type=Stop Tif=Gtc Oco='-ee5e7d033c7a43338f4214934d22eebb' Filled=100000 Fill price=0.851 Token='101c81a8668e4359af48578a7ceb987e' Gtd='01/12/2099 00:00:00'' generated by strategy 'ThreeDucks002/ee5e7d033c7a43338f4214934d22eebb' : This strategy will be disabled and NinjaTrader will attempt to cancel/close any strategy generated orders and positions. Please check your account orders and positions and take any appropriate action.
**NT** An over fill was detected on order 'Order='812667891/U618350' Name='Exit1Stop' State=Filled Instrument='$EURGBP' Action=BuyToCover Limit price=0 Stop price=0.8509 Quantity=0.1M Strategy='ThreeDucks002' Type=Stop Tif=Gtc Oco='-18a6574af0c0426ab658783be27c04c6' Filled=100000 Fill price=0.85098 Token='ce5614a76e604012a998f8d4db2138f4' Gtd='01/12/2099 00:00:00'' generated by strategy 'ThreeDucks002/18a6574af0c0426ab658783be27c04c6' : This strategy will be disabled and NinjaTrader will attempt to cancel/close any strategy generated orders and positions. Please check your account orders and positions and take any appropriate action.
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 Execution BuyToCover [email protected] STP (Filled)
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 OrderUpdate Flat BuyToCover [email protected] LMT (Cancelled)
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 Execution BuyToCover 100000 MKT (Filled)
**NT** Disabling NinjaScript strategy 'ThreeDucks002/18a6574af0c0426ab658783be27c04c6'
**NT** Disabling NinjaScript strategy 'ThreeDucks002/ee5e7d033c7a43338f4214934d22eebb'

What can I do from here?

Thanks

[NinjaTrader_Brett]

Hello,

This can be seen here:

2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 Execution BuyToCover [email protected] STP (Filled)
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 OrderUpdate Flat BuyToCover [email protected] LMT (Cancelled)
2010-09-28 12:25:00 ThreeDucks002 $EURGBP LS tf:5Min Q:100000 Execution BuyToCover 100000 MKT (Filled)

Looks like in the same microsecond a stop order was filled when a MKT order was submitted and both where filled at the same time. you sent your traces into Bert and Bertrand responded to you that he was going to test this and get back to you. I will leave this open for bertrand to look into as it looks like you even got an overfill on the overfill protection. How many stop orders are you submitting to the market at one time with the strategy?

Does this only occur with one strategy? Or do you have multiple strategies this is occuring on?

I look forward to assisting you further.

[adamus]

Initially I thought it was my strategy code but I have the strategy running on a live account and a simulation account at the same time, for 11 forex pairs.

So far both overfills have been on EURGBP.

The first overfill was only on the simulation account and the live account functioned as planned, i.e. it exited its position on the stop and the OCO limit was cancelled. In the simulation account, this extra market order was submitted.

The second overfill happened simultaneously on both the live and the simulation account.

I shall check the code to see where and why I would submit market orders. It may be a reversal technique.

[adamus]

Argh! Why couldn't it just be a simple mistake in my script? That would have been too easy.

Unfortunately I don't use market orders in this strategy. It was another one I was thinking about which has position reversals. Nor do I label any of my orders "Close". The order labels are all "EntryXxxxx" or "ExitXxxxx" where xxxx = something useful.

So I'm afraid I can't see how any of the orders my strategy placed could have morphed into those market orders.

[NinjaTrader_Bertrand]

adamus, when IgnoreOverFill is not set to 'true' going unmanaged, the strategy would terminate itself if an overfill is detected, as part of this process it would attempt to close it's open position, hence you see the 'Close' market order being issued.

[adamus]

adamus, when IgnoreOverFill is not set to 'true' going unmanaged, the strategy would terminate itself if an overfill is detected, as part of this process it would attempt to close it's open position, hence you see the 'Close' market order being issued.

But this is just the plain old managed order scenario.

Going forward what I see as the crucial discussion point here is whether that overfill was caused by my bad script or it was generated by NT7. I would love to find the error in my script because I could fix it. But it seems that it's you guys who have to fix something.

The way I see it, there were two orders in the market, an OCO bracket, one stop and one limit. The stop got hit and executed, the limit got cancelled. So what caused the overfill?

I answered Brett's questions above.

For my part, I'm going to put more log statements in my order submit clause.I'd be interested to know if you can make any progress analyzing the situation too.

What bugs me is that the market order reverses my position. If it were really my strategy causing an overfill through poor order management, and the market order was meant to 'fix' the overfill, where is the actual overfill that it's trying to reverse? It's not the stop order execution - that's the normal end of the trade.

[NinjaTrader_Bertrand]

Thanks for clarifying Adam, will recheck your logs submitted for this case - ultimately it would boil down to run a reproducible scenario to closely investigate. The strategy termination behavior would just be attempting to flatten it's strategy position, it would not try 'fixing' the overfill per se (default behavior in managed mode and with enabled RealTimeErrorHandling is the strategy termination).

[adamus]

I took my eye off the ball on this issue over the last week but just this morning I have had two overfills, fortunately both on the IB simulated trading account, shortly after I re-enabled the strategies there.

I have sent in my logs. TracesAndLogs.zip

I still don't have a clear idea of what you are trying to tell me about the last overfill situation, i.e. I don't know why it placed a market order which I had to take action to sort out.

This time I think both overfill situations are different.

On GBPUSD, there was one order at 11:53 and

EDIT: Sorry I'm talking rubbish here:
""in the Orders tab it was for 150000 but in the execution tab it was 300000. I will attach images. ""
It's the position that's double. This is all too far gone for me to work out what was going on. Unfortunately I just closed NT to try to get it to sort out its idea of what forex positions I've got, because it is way out of whack at the moment. But by closing it I lost my output window logging so I now can't tell what the sequence of order was that the strategy thought it was doing.

For EURJPY, I don't see anything wrong - apart from the presence of the error.

[NinjaTrader_Bertrand]

Adam, I just responded to your note sent to support - we'll need to review the logs closely and get back to you, it looks to me as if you're dealing with a sync issue here.

[adamus]

Bertrand, another log to review

http://www.ninjatrader-support2.com/upload/20101005_446881fec34a4c179a2383893a5f9af4_TracesAndLogs.zip

What happened this time - AUDUSD just closed out a trade. I can't see any sign of an overfill.

I am beginning to suspect that the overfill detection is based on NT's idea of the position, which could be bad news as it looks like the position is the first thing that NT loses track of when things start to go wrong.

[NinjaTrader_Bertrand]

Thanks Adam - the overfill condition is not position tied, it would occur if you send a cancel request, but the order comes back either partfilled / filled, thus you got an unexpected fill for it and hence it was overfilled comparing to the expected outcomes.

[adamus]

Do you mean that overfills are a random but constant threat?

Is there no error handling that I could do? Or is that something that can only be done in unmanaged mode?