Live Position Tracking Error

[cgeorgan]

Guys-

I can't help but get the feeling that there's something fundamentally flawed about position tracking ("the database").

For some background: I've been testing strategies now for almost 6 weeks on NT. In those 6 weeks, every single simulator account that I've used (at least 4), have become "corrupted". By that, I mean that the actual positions are irrecoverable, no matter what procedure used to reconcile them. This includes "flattening" by trades, adding/removing executions, disconnecting from the data source to synchronize the DB, etc. Ultimately, I've had to move to a brand new account to continue tracking results.

I figured I'd have more success when NT was trading live. So here I am, using NT + Trading Technologies...things were going well the past couple of days, but when I turned NT's strategy on last night @ 6:29 PM CST (after closing out a long position at 3:50 PM EST), I got some *really wierd behavior*.

NT Now believes that I am long 7 contracts of 6A. You can see from the attached photo ("6A TT") that as soon as I connected, TT attempted to sell, then buy, then sell, then buy again, a total of 7 contracts each time in the span of literally NO seconds.

I do have "sync positions = true" set in my strategies - so I believed that perhaps what I was seeing in Trading Technologies was wrong (that I was not in fact flat). I called the clearing department to confirm, and I was in fact flat.

Yet no matter what I did to "flatten" Ninja's database, it still believed that I was long (or, at various times, short). To answer your questions, I'm pretty good with Ninja now and I did every single thing possible to fix this. This includes:

• Turning synch off
• Disconnecting from data sources and repairing the DB (didn't fix)
• Entering a manual trade to "flatten" Ninja (didn't work).
• Stupidly trying to "trade it flat" in TT (I made some money doing this but didn't fix the NT problem).
• Turning my whole computer off, restarting everything, etc.

As you can see, I've attached 3 shots:

• A shot of my executions, showing a phantom entry on NT's part @ 6:37:55 "NT Time" (I say that because nothing on the executions for 6A match up with TT executions), resulting in a long position of 7 6A's and marked "external".
• A shot of my TT screen, showing that I am in fact flat (yes, flat since the session's close of 4PM CST, as defined by my session manager - which states that the trading day runs from 5PM Sunday to 4PM Monday, 5PM Monday - 4PM tuesday, etc).
• A shot of my NT position screen, showing how it thinks I'm long 7 contracts.

I'm not sure what to do next here. When you guys write me, I'll shoot you my logs and trace files.

In the end, I've said it before; There needs to be a better way for users to get into NT's database to adjust positions. The current procedures just aren't enough. When somebody who's dedicated months to using NT (a technologically proficient individual) cannot fix a problem like this, there's a problem with the program - not the user.

[cgeorgan]

Quick reply-

I'm getting on my clearing firm here; I might in fact have this position. Highly frustrating...

EDIT:

Guys - with regards to one of the photos I posted (6A error, the "executions" tab I posted), I need to know exactly what the trade @ 6:37:55 PM on 8/30 was - the entry of 7, marked "External". How can I find where this was entered into?

[cgeorgan]

Here's what I'm getting - this appears to be some sort of actual, serious problem.

This is what my log file is stating happened:

8/30/2010 6:37:27 PM|1|2|Advantage Futures LLC: Primary connection=Connecting, Price feed=Connecting
8/30/2010 6:37:35 PM|1|2|Advantage Futures LLC: Primary connection=Connected, Price feed=ConnectionLost
8/30/2010 6:37:35 PM|1|2|Time to flatten=12:00:00 AM, Flatten all=Disabled
8/30/2010 6:37:55 PM|1|32|Order='090AJX002/737A6456' Name='External' New state=Accepted Instrument='6A 09-10' Action=Buy Limit price=0.8922 Stop price=0 Quantity=7 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
8/30/2010 6:37:55 PM|1|32|Order='090AJX002/737A6456' Name='External' New state=Working Instrument='6A 09-10' Action=Buy Limit price=0.8922 Stop price=0 Quantity=7 Type=Limit Filled=0 Fill price=0 Error=NoError Native error=''
8/30/2010 6:37:55 PM|1|32|Order='090AJX002/737A6456' Name='External' New state=Filled Instrument='6A 09-10' Action=Buy Limit price=0.8922 Stop price=0 Quantity=7 Type=Limit Filled=7 Fill price=0.8902 Error=NoError Native error=''
8/30/2010 6:37:55 PM|1|16|Execution='1g8rs737pvxut' Instrument='6A 09-10' Account='737A6456' Exchange=Globex Price=0.8902 Quantity=7 Market position=Long Operation=Insert Order='090AJX002' Time='8/30/2010 6:37:55 PM'
8/30/2010 6:37:55 PM|1|64|Instrument='6A 09-10' Account='737A6456' Avg price=0.8902 Quantity=7 Market position=Long Operation=Insert Currency=UsDollar
8/30/2010 6:41:43 PM|1|128|Enabling NinjaScript strategy 'CRG_Momentum_MultiTimeFrame2_Fix/c68ecc9bdb284729a776796abbf66206' : On starting a real-time strategy - StrategySync=WaitUntilFlat SyncAccountPosition=True

Background: I have confirmed with Advantage Futures (clearing firm), as well as the CME, that I was flat at the close last night, close being 4PM CST (all times are CST here).

You can see that - as soon as I connected to Advantage and my account, I received indication of an "external" fill that had not registered anywhere on my TT screen or Gateway, long 7 contracts.

Next, you can see that I started my Ninja Strategy, which attempted to sell this position to get flat. At that point, in TT, I saw that I was in fact SHORT 7 from the execution.

The entire debacle continued all night, whereby each time I thought I was flat, Ninja thought I was long 7, and each time I sold this "long" position in NinjaTrader through TT, TT would tell me I was short 7.

[cgeorgan]

Quick update

Guys-

Seems more and more like this is a TT/Clearing firm issue, not a Ninja issue. If it becomes the latter, I'll let you guys know.

CG

[NinjaTrader_Jason]

Hello cgeorgan,

Sounds good, please let us know if the issue is not a TT/clearing firm issue.

Orders can be listed as 'External' in the following circumstances.

When NinjaTrader submits an order, it maintains a record of this in its database. When an order/execution event is received from the connected broker, NinjaTrader will mark this as "External" if it did not already have a record of this order in the database.

There are two scenarios where this could happen:

• You submit an order through NT
• Drop/Reset the Database
• NT receives an order event and since the database is in a "fresh" state, it would mark the order as external

OR

An order was submitted to your account outside of NinjaTrader such as by your broker or through another trading application.

[cgeorgan]

Sounds good guys -

What it's looking like is that the clearing broker rolled the TT date at 3:15 CST, despite trading ending at 4PM CST. I was in a long position of 7 between 3:15 CST and 3:50 CST, when that position was closed.

On restart, it appears the gateway informed me I was still long 7, even though I was flat. Ninja attempted to flatten this position, selling 7...making me think I was short, causing me to buy.

Essentially, I was engaged in:
Short, cover, short, cover, short cover.

When in reality, it was: Long, sale, long, sale, long, sale, long.

[LotsaOptions]

Solution?

Last month I had a live phantom position that I couldn't get rid of labeled "External" that my broker assured me I wasn't in. Though I haven't puzzled out exactly how it got there I suspect the same way as cgeorgan's phantom position.

To remove the position, after trying the same things cgeorgan tried, my broker (Optimus/Vision) sent a phantom execution to sell a long position to trick NT into thinking it was flat again.

This happen to me using NT 6.5, but I'm in the process of transitioning to NT 7 (which is AWESOME, by the way). Is this something that could also happen on NT 7?

What's the solution to this? If it happens again is there any way to remove specific positions from the NT database without having to contact the broker? Would syncing up the broker's rollover time with a session template prevent this from happening again?

[cgeorgan]

Last month I had a phantom position that I couldn't get rid of labeled "External" that my broker assured me I wasn't in. Though I haven't puzzled out exactly how it got there I suspect the same way as cgeorgan's phantom position.

To remove the position, after trying the same things cgeorgan tried, my broker (Optimus/Vision) sent a phantom execution to sell a long position to trick NT into thinking it was flat again.

So, what's the solution to this issue? Would syncing up the broker's rollover time with a session template prevent this from happening again? And if it does happen is there any way to remove specific positions from the NT database somehow?

I'll share with people what happened, as far as I could discern after much deliberation with the CME, my back office, etc.

It turns out that our clearing broker actually rolls the trade date for *every* instrument at 3:15 CST. I was trading the Sep. 6A contracts, which technically don't close until 4 CST. I was in a long position of 7 after 3:15, which I closed out for the day before 4.

When I restarted, Ninja was telling me (via the TT Gateway) that I was actually long 7 contracts of 6A. This did not square with Ninja's own tracking, nor with my tracking, which told me I was flat to the contract.

Anyways, this went back and forth all night and to the next day, when the CME finally told me that I was still long 7 contracts (somehow, my sale of 7 - for which I had received confirmations, etc - went totally missing). I was forced to sell out of these contracts, literally at the low of that day. Cost me $3k.

[NinjaTrader_JoshP]

LostOptions,

If you wanted to offset a phantom execution that is coming through from your brokers end you can go into the Executions tab of the Control Center, right click, Add Execution and just put in a competing offset execution to cancel it out.