Wrong average price

[RobotSyndicate]

Hello,

When I enter a live position with IB (and this has happened using market order, limit orders, via a strategy, and also entering a position manually...) NT will occasionally display a drastically incorrect position average price. For example, I will use chart trader and click to buy 5 NQ contracts with a market order, it will sometimes say my entry price is HALF of what it actually is. Like instead of 13000, it will say 6500. Or sometimes it will say 0. Or sometimes other way off values. This occurs every day, but I am placing hundreds of trades a day, so its probably around 2-3% of my trades. This incorrect value is then used by NT to display incorrect P/L, etc etc etc. I am using NT 8.0.23.2 and this has occurred both with IB TWS 978.2c and IB Gateway 978.2c. I have tried using Kinetick for historical and live data, and I have also tried using kinetick for historical and IB for live data. It happens either way. This occurs whether I apply a strategy to a chart, or run it from the strategy tab. The price of a stock could be 100, I will set a live limit order at 95, my limit order will fill with IB, and it will literally give me a position with an average price of 0 or something, regardless of my actual average price in IB.

What could be causing this? Thanks!

[NinjaTrader_Jim]

Hello RobotSyndicate,

Thanks for your post.

I have moved this ticket to the Platform Technical Support forum as you mention the symptoms are something that can be encountered with manual trades and not specific to strategy trades.

You mention this only happens 3% of the time. What is unique about these trades where you are getting unexpected PnL/fill price reports? We would like to reproduce such trades or similar so we can test and review on our end. When I test on my end submitting orders to our IB paper trading account, I see average fill prices that I would expect. Are these affected orders partial filling or is there something else we can identify with them that can make this symptom testable? (screenshot of my Orders tab attached.)

Could you also be specific as to where exactly you are seeing the incorrect information? For example, is it just the order fill price as seen in the Orders tab, or do you see other affected items like in the Accounts tab of the Control Center?

This may be easier to achieve with manual trades following some steps, but if you feel it is easier to demonstrate through a strategy, please feel free to attach a strategy that can reproduce the symptoms taking minimal steps with other unrelated code removed.

I look forward to assisting.

[RobotSyndicate]

Hi Jim,

I will take a look at things this coming week and see if I can figure out what is unique about the trades that have the error. I see the incorrect price as my average fill price on the chart if chart trader is on (it will literally put my fill price line at $0 and instantly show a massive unrealized gain or loss), in the positions tab, and in the strategy tab as unrealized P/L. I haven't actually checked the orders tab to see if it lists an incorrect fill price there, I'll look for that next time it happens. One interesting thing is that if I complete the trade, then the actual P/L will correct itself right away. Like if I enter a long position at $100, the average fill price may incorrectly state $0 and display a HUGE unrealized gain, but then if I sell the position at $101, then it will correctly display $1 realized profit per share/contract. But many of my strategies use AveragePrice in determining stop losses and profit targets, so its a problem...

Anyway, I'll try to figure out some unique features of the trades if possible. The way I see it the most is as I run the same strategy on the entire S&P500, and it makes about 300-600 trades/day, and this incorrect fill price will occur with about 10-15 trades per day. And it will happen with a particular instrument for one trade, then not happen with the same strategy on the same instrument in the same day with no changes in connection and without restarting the strategy in between.

Thanks again for your help, I'll let you know what I can figure out next week!

[RobotSyndicate]

Ok I got some more info. Today I used market orders from chart trader with NQ futures live with IB gateway to get the error. It occurred with both partial fills and complete fills. Actually today with NQ futures it happened every single time I tried. In the order and execution tab/log it will display the CORRECT entry price. But on the chart via chart trader and in the positions tab it will display the incorrect price. Also, I noticed that in the executions tab it displayed the incorrect position quantity, but everywhere else it displayed the correct position quantity. For example, I would go long 5 NQ contracts, and in chart trader and in the positions tab it would say 5 long. But in the executions tab it would say 18. If I disconnect from IB and then reconnect, it does not fix the error. If I disconnect from IB and kinetick, close IB gateway, restart IB gateway, and reconnect to IB, then it does fix the error, until it happens again. When it happens next I'll try and see if just disconnecting/reconnecting from kinetick with fix it, in case the IB gateway restart isn't needed. Here are some screen shots...

[RobotSyndicate]

I just placed another live IB market order on NQ this morning, and took more screen shots. You can see that it shows a price of zero on chart trader (and also in the positions tab, with associated incorrect unrealized profit), but when I zoom in on the same chart, it places the trade markers on the correct spot and prints the correct entry price.

[RobotSyndicate]

Ok now I closed out that position via chart trader market ordrer, and then I placed a 5 contract market sell short order and it filled, and it shows the correct price everywhere, BUT I see that in the executions tab it still shows an incorrect position. It seems to think that I still have some long NQ contracts, instead of just being 5 short. The positions tab shows the correct quantity and marketposition, and so does chart trader, but the execution tab does not.

[NinjaTrader_Jim]

Hello RobotSyndicate,

I've been trying several order submissions on our Interactive Brokers paper trading account and I cannot reproduce these symptoms.

As you are able to get these unexpected positions cleared when shutting down IB Gateway and reconnecting, we may be dealing with an issue involving position reporting from IB Gateway, but we should also eliminate any potential database issue that may be contributing.

Is it possible to recreate these symptoms after renaming the database file in NinjaTrader? We see that there are other positions recorded in the Executions tab, which may then contribute to the position reported with the executions. When you check these executions are these all accurate and did they actually take place on your account? If some executions/positions are mixed up in the database file, we may see symptoms like this.

The database file is located here: Documents\NinjaTrader 8\db\NinjaTrader.sdf. You may close NinjaTrader, and rename this file to NinjaTraderOld.sdf. After restarting NinjaTrader, we will have a new database file.

If we can reproduce here, is there anything unique I should do on my end? I have been submitting Buy market and Sell market orders for a quantity of 5, and the entry prices and quantities are what I would expect to see.

I look forward to hearing from you.

[RobotSyndicate]

Hi Jim,

All of the executions that are listed in the executions tab are occurring in the account, at least for the last two days that I looked back. I've actually been experiencing this for a few weeks. And I've tried recreating the database, reinstalling ninjatrader, and recently for other reasons I put the whole system on an Amazon EC2 cloud instance, and after doing any of these the errors go away for a few days, until they come back again. I just renamed the database again and restarted NT. And I'll just let the things run without any coding or compiling or other mucking around, and we'll see if it still happens. Something interesting that I have noticed, is that it seems that after I reinstall or recreate the database, everything works until a particular instrument starts having the error. Then that particular instrument will keep having the error (not every time, but frequently) but other instruments will not. Then over time I will get more and more instruments acting up. Like over the past few days, NQ would give me this error about 50% of the time, but I placed dozens of trades each in ES, GC, HG, and SI with no errors at all on those instruments.

I'll create a simplified version of my strategy that blasts orders all over the S&P and see if I can recreate it on a paper trading IB account.

[NinjaTrader_Jim]

Thanks RobotSyndicate,

I was thinking the same, if we can get the issue to come up from a clean database using a [simplified] strategy to make the symptoms more likely to encounter, we should be able to reproduce on our end.

I look forward to performing any further tests.

[RobotSyndicate]

Hello,

As I work on that, here is another thing I noticed. I added code to print the strategy position and account position so I could monitor it to see if they are correct, because of the incorrect position sizes being displayed in the execution tab:

Print("ES Strategy Position: " + Position.Quantity);
Print("ES Account Position: " + PositionAccount.Quantity);
Print("ES Strategy Direction: " + Position.MarketPosition);
Print("ES Account Direction: " + PositionAccount.MarketPosition);

And within an hour or so I saw a strategy use a limit long order to buy 3 ES contracts. At that moment the strategy disabled itself (I'm not sure why yet). Then my IB account still had the 3 long ES that the strategy bought, BUT my account position displayed in NT in the strategy tab and the positions tab said zero. So when I enabled the strategy (it adopts account position), it enabled assuming an account position of zero, instead of the true account position of long 3 ES contracts. Is there a certain event that has to occur for NT to update the true account position? How could this happen?

Thanks again!

[NinjaTrader_Jim]

Hello RobotSyndicate,

Strategy positions are based on Execution events while Account Positions are based on PositionUpdate events. (Strategy OnPositionUpdate is driven by PositionUpdate events which are filtered to the strategy's actions.) NinjaTrader will need to see these events to update the position.

If you would like to send me log and trace files, and note the exact time the account position was not properly updated, we could see if there is an issue with the ordering of these events or if a PositionUpdate was missing. If that is not the case, we would still need a strategy that we can use to encounter the symptoms.

You may send this information to me at scriptingsupport [at] ninjatrader [dot] com with the text "Attn Jim 2985801" since we are now looking at this from a strategy perspective. Please also include a link to this thread. Log and Trace Files are located in the Documents\NinjaTrader 8\log and Documents\NinjaTrader 8\trace folders.

I look forward to hearing from you.

[MrHump]

Hello RobotSyndicate,

Did you ever manage to resolve the above? As I am seeing similar issues on my simulation account!

Thank you in advanced