stop loss and profit order not submitted sometime by NT 7

[hongchen2000]

Hi,

Does anyone experience this problem?

I run a simple automated strategy and about 1 out of 10 trades, NinjaTrader would not submit the stop loss and target order as expected. It has caused me a few losses because I can't babysit my trades all the time.

I looked into the trace file for the troubled trade and I found the following message:

2011-06-17 10:58:05:656 Cbi.Globals.
ProcessEventArgsInThreadContext, exception caught: Object reference not set to an instance of an object.: ExecutionUpdateEventArgs: Execution='MOS/0000d3eb.4dfb1a5a.01.01' Instrument='MOS' Account='U812429' Exchange=Arca Price=61.12 Quantity=18 Market position=Long Operation=Insert Order='685029527' Time='2011-06-17 10:58:05 AM'

To debug the problem, I have simplified my strategy to something like, and yet the problem persists:

OnBarUpdate() { if (position== flat), DoEntry(); }

DoEntry() { if (EntryConditionOK()) { SetStopLoss(...); SetProfitTarget(...); EnterLongStopLimit(); }

Does anyone has any insight in this problem?

So far, I have not received any intelligent response from NT support.

[NinjaTrader_Bertrand]

hongchen2000, for the exception you posted from the trace - please retry the same strategy after using a fresh database file. You could do this step by shutting NT7 down and going to Documents\NinjaTrader 7\db and rename the present NinjaTrader.sdf file to for example NinjaTraderOld.Sdf, then restart NT7 and retry your scenario. Please let us know how it goes.

Thanks,

[hongchen2000]

Hi, Bertrand,

I just did that and it seems that all my changes to the db is lost. (mostly manual edit to add additional exchange associations to stock, due to IB error).

But I will try this change, since it is the first somewhat meaningful suggestion I got from NT. (Although I still have doubts since the I have seen stopLoss/profit order not consistently submitted for the same symbol on the same day).

It would be nice if you tell me the rational behind your thinking in asking me try this change.

[NinjaTrader_Bertrand]

The rational is that from the trace seen I feel your database file has experienced a corruption unfortunately, it could be a partial one, so not all instruments you trade would be affected - which would make sense from the issues you see. The IB exchange changes are saved to the DB as well, correct - for this I would recommend at least weekly backup in the future so you can easily restore your work environment.

[hongchen2000]

This is the kind of response I expect from NT, with tangible intelligence behind the thinking! Whether this turns out to be the cause of the problem is a different issue and we will see in the next few days of trading.

What I hate the most is those "If there is a problem, it must because of your program is too complicate" responses.

[NinjaTrader_Bertrand]

You're welcome, please let me know what you see then going forward. Whenever there's custom code in the mix we have to isolate and simplify to make approaching the issue feasible so variables are excluded - for this we normally ask to break apart the code into it's simplest form and then see if the issue could still be reproduced.

[hongchen2000]

Today there is no missing stoploss/profit order. Although it is only one day but that is a good sign.

[hongchen2000]

Hi, Bertrand,

It has been a whole week without the missing stoploss/profit target issue in live trading. It is almost certain that resetting the DB helped fix this issue (where it happens almost daily before).

Thanks a lot for the solution.

However, there is another issues that still happening. The issue is the under-calculation of stopLoss order quanity (and sometime profit target quantity too), when the entry order was filled in multiple executions(several partial fills).

This happened again this Thursday with FSLR. Summary of the event sequence as below:

1)Short sell FSLR 140 order was executed with three partial fills(100+38+2).
2)Profit target order was submitted for 140 shares
3)However StopLoss order was submitted for only 138 shares.
4)StopLoss order was triggered and executed for 138 shares.
5)I had to manually sell the missing 2 remaining shares when I discovered the missed shares.

This issues happened before, and the missing shares could be 100 shares or more sometimes. While most of the time the under calculation is for stopLoss orders but it also happened for profit order (or both profit order and stoploss order).

Any idea how why this happened?

[hongchen2000]

2011-06-30 1:27:02 PM|1|32|Order='a249d75d58b541349f8de4b322af9136/U812429' Name='SE1_Flag303' New state=PendingSubmit Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=120 Type=StopLimit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:27:02 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=Accepted Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=120 Type=StopLimit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:01 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=PendingChange Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:01 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=Accepted Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:06 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=Working Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:13 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=PartFilled Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=100 Fill price=136.52 Error=NoError Native error=''
2011-06-30 1:28:13 PM|1|16|Execution='FSLR/00012bb9.4e0c7add.01.01' Instrument='FSLR' Account='U812429' Exchange=Default Price=136.52 Quantity=100 Market position=Short Operation=Insert Order='1761843858' Time='2011-06-30 1:28:13 PM'
2011-06-30 1:28:14 PM|1|32|Order='9c0980c700cb4d29bfd733d206f8050e/U812429' Name='Stop loss' New state=PendingSubmit Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=100 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1aa0abf326da46d9bef525ccf16c5bda/U812429' Name='Profit target' New state=PendingSubmit Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=100 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|2|4|Order Event Warning:Attribute 'Outside Regular Trading Hours' is ignored based on the order type and destination. PlaceOrder is now being processed. 2109
2011-06-30 1:28:14 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=PartFilled Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=138 Fill price=136.52 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|16|Execution='FSLR/00012bb9.4e0c7ade.01.01' Instrument='FSLR' Account='U812429' Exchange=Default Price=136.52 Quantity=38 Market position=Short Operation=Insert Order='1761843858' Time='2011-06-30 1:28:14 PM'
2011-06-30 1:28:14 PM|1|32|Order='1761843858/U812429' Name='SE1_Flag303' New state=Filled Instrument='FSLR' Action=SellShort Limit price=136.52 Stop price=136.53 Quantity=140 Type=StopLimit Filled=140 Fill price=136.52 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|16|Execution='FSLR/00012bb9.4e0c7adf.01.01' Instrument='FSLR' Account='U812429' Exchange=Default Price=136.52 Quantity=2 Market position=Short Operation=Insert Order='1761843858' Time='2011-06-30 1:28:14 PM'
2011-06-30 1:28:14 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=Accepted Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=100 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=PendingChange Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=PendingSubmit Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=100 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Accepted Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=100 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Working Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=100 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=PendingChange Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=140 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=PendingChange Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=Accepted Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Accepted Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=140 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:14 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Working Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=140 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:28:15 PM|1|64|Instrument='FSLR' Account='U812429' Avg price=136.5151744 Quantity=140 Market position=Short Operation=Insert Currency=UsDollar
2011-06-30 1:33:10 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Working Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=101 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:33:10 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=Accepted Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=38 Fill price=136.71 Error=NoError Native error=''
2011-06-30 1:33:10 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=Working Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=38 Fill price=136.71 Error=NoError Native error=''
2011-06-30 1:33:10 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=PartFilled Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=38 Fill price=136.71 Error=NoError Native error=''
2011-06-30 1:33:10 PM|1|16|Execution='FSLR/0000f073.4e0cf031.01.01' Instrument='FSLR' Account='U812429' Exchange=Default Price=136.71 Quantity=38 Market position=Long Operation=Insert Order='1761843860' Time='2011-06-30 1:33:10 PM'
2011-06-30 1:33:11 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Working Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=0 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
2011-06-30 1:33:11 PM|1|32|Order='1761843860/U812429' Name='Stop loss' New state=Filled Instrument='FSLR' Action=BuyToCover Limit price=0 Stop price=136.7 Quantity=138 Type=Stop Filled=138 Fill price=136.709 Error=NoError Native error=''
2011-06-30 1:33:11 PM|1|16|Execution='FSLR/0000f073.4e0cf032.01.01' Instrument='FSLR' Account='U812429' Exchange=Default Price=136.709 Quantity=100 Market position=Long Operation=Insert Order='1761843860' Time='2011-06-30 1:33:11 PM'
2011-06-30 1:33:11 PM|1|32|Order='1761843861/U812429' Name='Profit target' New state=Cancelled Instrument='FSLR' Action=BuyToCover Limit price=133.85 Stop price=0 Quantity=0 Type=Limit Filled=0 Fill price=0 Error=NoError Native error='Order Canceled - reason: (202)'
2011-06-30 1:33:12 PM|1|64|Instrument='FSLR' Account='U812429' Avg price=136.5151744 Quantity=2 Market position=Short Operation=Update Currency=UsDollar
2011-06-30 1:41:09 PM|1|32|Order='1761843862/U812429' Name='External' New state=Working Instrument='FSLR' Action=Buy Limit price=136 Stop price=0 Quantity=2 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''

[NinjaTrader_Bertrand]

Good to hear the fresh DB helped you out - what NinjaScript methods are working with for your stoplosses and profit target codes? Can you reproduce this on replay data? If you use custom logic here, it would mean you have to debug your code to understand exactly why this error is seen in those scenarios.

[hongchen2000]

It doesn't happen when tested on historical data.

It is the same simple strategy I mentioned earlier on this thread. (all code in OnBarUpdate(), and calculatedOnBarClose is true):

OnBarUpdate() { if (position== flat), DoEntry(); }
DoEntry() { if (EntryConditionOK()) { SetStopLoss(...); SetProfitTarget(...); EnterLongStopLimit(); }

My suspicion is some kind of race condition here happening when multiple fills happen and NT trying to update the new quantity for stopLoss and profit orders.

I looked into the trace file and didn't find any exception or error. It seems like the last fill of third partial fill event was ignored by stopLoss quantity calculation or the calculation was tryingt to update/overwrite the same variable?)

(in the log, the profit order quantity updated once, from 100 to 140, stop loss also updated twice, from 100 to 138).

[NinjaTrader_Bertrand]

Ok, I understood it does not happen in backtesting - but what about Market Replay then? This would run off recorded Level 1 / Level 2 data and can provide further insights into strategy behavior when tested:

http://www.ninjatrader.com/support/helpGuides/nt7/market_replay.htm

If you could direct the log and traces from the day of the issue again to me that would be very helpful.

Thanks,

[hongchen2000]

It has been almost one full month of trading and the issue resurface only once.
(The only time that happen involves a partial fill of 2 shares and I have since eliminate that small shares execution and everything is fine now. )

Thanks for the help in fixing the db to resolve the issue.

[NinjaTrader_Bertrand]

Thanks for letting me know and reporting back here.

[hongchen2000]

The problem happens once again today. This time is with SPY. And now I see the pattern of this problem. It happens(although not always) when 1) an order is executed by multiple fills and each fill has fractional shares. For example, today's SPY order of 120 was executed in two filles: 44 shares and 76 shares. And the generated stop/profit order is only for the first 44 shares.

I don't have replay setup so I can't try to simulate the execution. And I don't think sending you log/trace file will help you in anyway to debug this problem, just as last time.

So my (strong) suggestion to you is to incorporate a test case in your development for order that has multiple fills and with fractional shares with each fill. I know this is not an ease test case to setup and most likely you have not tested that. However, to claim you have a high quality execution platform, it is better to make even this infrequent cases working correctly. Because in today's fast moving market, one hit on this bug would be enough to cause devastating damage to one's portfolio.

Thanks.