Buy/BuyToCover Position Problem

[heech]

Hi,

Ray accidentally deleted the previous thread. I'm trying to capture the last few messages here. I hope someone from NT is looking at this problem seriously, not just trying to keep me occupied on the forums.

Dierk,

I'm definitely not good.

I actually started with a brand-new system over the weekend, meaning completely empty database. All of my strategies are running well... the positions tab and strategies tab all show the correct strategy/actual position. (Long in some instruments, flat in others.)

But the execution tab is still showing bad news. I had some ExitLong() orders this morning, and execution tab is showing these sells as "Entry", and my resulting position as "short".

One example:
CENX Sell 1,300 7.50 12/1/2008 6:35:07 AM 4510166425 Entry 1,300S 4510165240 Sell 0 1 707028387 Izone

My strategy shows the position at this point as flat, but execution doesn't agree. I'm not positive (because it hasn't tried yet this morning), but I strongly suspect when I try to go long... it will issue a BuyToCover command, and it will choke again.

So, whatever this problem is, I don't see how it has anything to do with the database.

You will keep having problems as long as Executions tab/Position column does NOT show "-" (flat), but you are in fact flat in the Accounts tab. You then need to reset your DB as per my post below.

It does not make sense to test anything unless you don't have this well defined starting point.
__________________
Dierk, NinjaTrader Customer Service
NinjaTrader is a FREE application for advanced charting, market analytics, system development and trade simulation.

View schedule of upcoming online product training events.

Dierk,

I'm not sure what you are suggesting I do. I had an *empty* database before the executions started this morning. And as soon as a stop order filled, they came in as Entry/Short.

Are you saying I should terminate my strategies, reset the database after every order is filled?

And can you just confirm for me that you understand what I'm saying? That the Strategies/Positions column does show flat?

Quote:
Originally Posted by heech
Strategies/Position: CENX, - (flat)
Positions: nothing (for CENX)
Execution/Positions: Sell Stop, Entry, 1300S.

This means your NT internal execution tracking still is out of sync. Please try Tools->Options->Misc->Reset DB to clear out your historical trade data (please see my post below). Please make sure your TDA account is flat as you reset your DB.
...

These 2 needed to be in sync (no matter if all are flat or all show the same ! position).
- easiest you sync them as your TDA account is flat (see below)
- if your account is not flat (or you would not want to flatten it), then you need to manually inject an execution on the Execution tab to sync them (Right click->Add execution)

Great, thank you! I am going to try the older version of NT first, and then I will give this a try. If that works, I'm a happy guy.

But again, because this is a persistent problem which happens after *every* restart, at the beginning of every morning... I really hope that doesn't mean I will need to manually inject an execution for every long position every morning. (How about a patched version of the software that looks at my long positions, and manually injects an "external" execution for each instrument at startup? That would make sense as a permanent fix.)

Am I really the only one that's having problems with holding instruments overnight?

Not sure why you think there is an overnight issue. If you synced your executions as per below, and just do nothing today and restart NT tomorrow, then everything will be in sync still.

I assume there's an overnight issue because I have yet to see a single "position" be correct going into the next day (according to the executions tab), out of the 20 stocks I'm trading. However, everything looks great the same day I acquired them. The original Entry/Long execution is absolutely there.

So whatever is happening, it's going missing overnight. And this is not happening occasionally... this is happening with 100% certainty, over 20 instruments, over 4 consecutive trading days.

If I understand you correctly:

- reset DB,
- manually add execution for each of my long positions to match the current actual (and strategy) position.

Wait, scratch that. I'm still confused. Or are you telling me to:
- manually add execution,
- and then reset DB?

And that should be it. I don't know why the original execution wouldn't have survived into the next day, but you believe my manual execution entry will survive... but what can I say, at this point I'm desperate, no one at NinjaTrader has given me a better idea. I can only try it.

And my last message, with my current guess as to the problem:

Dierk,

Based on your feedback, I went back to look at the actual database itself. (I submitted to you guys this morning; let me know if you don't have a link to the file, I'll PM it to you.)

Look at the 'nt_execution' table. There are a number of "virtual entry executions" for each of my instruments... I assume an attempt by NinjaTrader to do exactly what you suggest I do by hand, setup executions based on the long position it currently sees.

But the "provider" column between the virtual entry "long" execution and the actual sell execution do not match up! The virtual entry "long" executions are all provider 99, while the sell execution is provider 18. I have no idea what the provider represents... but could that be a problem? Why the long/short positions do not match up?

I wish someone there with access to the code would put in a few minutes to look into this issue, or hell, send me the relevant parts of the processing code (under NDA) to look at.

Ray replied to this saying "I might be right". I would really like to know what that means, and where we go from here. What is the provider column? What does it mean, how do we fix this?

[NinjaTrader_Ray]

My reply was not in context, disregard it. This is why I deleted it and accidentally deleted the thread instead of my post.

[heech]

My reply was not in context, disregard it. This is why I deleted it and accidentally deleted the thread instead of my post.

Ok, back to the drawing board.

Dierk, I'll try your suggestion with adding manual executions tomorrow. My fingers are crossed that it will work and "stick".

[heech]

Dierk,

Adding executions in order to sync with the database partially worked (yay?):

- sell orders today now show correct in the execution tab as "exits" and flat positions.
- but when I restart NinjaTrader, I received this message:

"Failed to read executions log file: The changes you requested to this table were not successful because they would create duplicate values in the index, primary key, or relationship. Change the data in the field or fields that contain duplicate data, remove the index, or redefine the index to permit duplicate entries and try again."

What should I do now? Reset or repair the database? Delete execution logs?

UPDATE: Doing a repair database seems to have worked! Great, I hope this lasts all the way through tomorrow... If it does Dierk, you're my hero.

[heech]

We're looking good, Dierk!

The one missing piece of information which I never understood is that NinjaTrader *needs* to know about the original execution... it's not sufficient for it to know about account versus strategy position (which is what the documentation talks about, which is also common to other platform)... NinjaTrader *needs* to have in its database an actual "entry" execution to match any long positions.

Without that matching execution, the state of positions elsewhere isn't sufficient to the lowest level order handling code. To me, this is definitely far from obvious, and I'd suggest making it very obvious in the documentation going forward. (Or, even better, fix the order handling code so it keys off of strategy Position rather than executions in the database.)

Dierk's instruction to "add" manual executions for all of the long positions I was carrying was the solution. I'm happier again... which just means I can now focus on the other 1000 issues still left on my to do list!

[NinjaTrader_JoshP]

Thank you for your suggestion. We will add it to the list for future considerations.