Strategy and Account Positions issues

[WhoKnows]

Had a double or triple mistakes today. I checked before starting up if there were any open positions, but I apparently missed that sometime during activating the strategies one EOD strategy assumed it had already gone short. I missed that this strategy started at 8.30. Ouch, you learn from expensive mistakes.
Bad start for second week NT live.

After sorting trough the log I have the following questions.


I have a basic EMA crossover resulting in a EnterLong and a separate exit ExitLong tied to the EnterLong via the name Entry_L. (and similar setup for the short side)
if (A) -> EnterLong(1,DefaultQuantity, "Entry_L");
if (B) -> ExitLong(1, Position.Quantity, "Exit_L", "Entry_L");
if (C) -> EnterShort(1,DefaultQuantity, "Entry_S");
if (D) -> ExitShort(1, Position.Quantity, "Exit_S","Entry_S");

1. If I use 'calculate at bar close=true' apparently condition B and C can be hit at the same time or during the same bar.
Apparently NT started 3 actions (I see 3 submits in the log): a.Exit the long Exit_L, b.Close position Long and c.Entry_S order
This resulted in a double position short, although 'entries per direction=1.
Is this is normal behavior? Can 'entries per direction' see the other orders if they result from the same bar?


2. I tried using the MarketReplay and added market position statements (removed them last week as I thought they replicated the effect of 'entries per direction=1'.
if (A) && Position.MarketPosition != MarketPosition.Long -> EnterLong(1,DefaultQuantity, "Entry_L");
if (B) && Position.MarketPosition == MarketPosition.Long -> ExitLong(1, Position.Quantity, "Exit_L", "Entry_L");
if (C) && Position.MarketPosition != MarketPosition.Short -> EnterShort(1,DefaultQuantity, "Entry_S");
if (D) && Position.MarketPosition == MarketPosition.Short -> ExitLong(1, Position.Quantity, "Exit_S", "Entry_S");
However I get still instances in marketreplay with the double positions when using 'bar close true'.
So far I can see with 'bar close false' appears to be ok.... but the strategy is not backtested in that state.

Is it then that when the 'Market.Position-rules' for Band C are called at the same time, NT does not yet see the result of the closing of the position via rule C and also executes rule B which puts me on 2 positions short?
If so why is this not prevented via the 'Entries per direction=1'?


3. I use Position.Quantity in 'ExitLong(1, Position.Quantity, "Exit_L", "Entry_L")' to make the strategy exit the right number of shares.
If I don't use this the strategy will sell 1 share only as it sees the 1 of the addes BarsArray timeframe as number of shares.
Is this the right way or are there simpler methods to convey the number of shares bought via DefaultQuantity? Can you e.g. use " " or empty space between ,,?


4. In the logfile I see that the term position is used in two ways.
Position in the 'category-column' apparently sees the AccountPosition.
The statement under the 'message column' states the Market Position, e.g. MarketPosition=Long. I assume this is the state the strategy calculated it is in.

I see e.g an instance under message ... Quantity=0 Market position=Long. I assume caused by my start problem. Is there a way to check this?
I would see this as a potential internal check that if Quantity=0 and MarketPosition is NotFlat .... there is obviously something very wrong.
Can I check for this and is so via which code?
e.g. If AccountPosition=0 and MarketPosition != MarketPosition.Flat, return.


5. I see several "warnings related to outside trading hours" in the log. I read in older forum posts this does not influence the execution.
Ok, but my PC is on US time and the instruments concerned are ETF's at ARCA, so all is well within the normal trading hours. So why the warnings?

6. I a1so found yesterday I had bought 100 shared of PKD. I could not remember if I put a low-ball long term bid in for this, but the log is very weird.... why does the second line refer to SLW instead on PKD??
Do these numbers tell you if this comes from NT or was started by TWS??

21/2009 10:50 Execution Execution='PKD/00011fb1.49773a9e.01.01' Instrument='PKD' Account='U523265' Exchange=Nyse Price=2.55 Quantity=100 Market position=Long Operation=Insert Order='554773528' Time='1/21/2009 10:50:25 AM'
1/21/2009 10:50 Order Order='554773528/U523265' Name='External' New State=Filled Instrument='SLW' Action=Buy Limit price=2.55 Stop price=0 Quantity=100 Type=Limit Filled=100 Fill price=2.55 Error=NoError Native error=''
1/21/2009 10:50 Position Instrument='PKD' Account='U523265' Avg price=2.56 Quantity=100 Market position=Long Operation=Insert Currency=UsDollar


A big list, thanks.

[NinjaTrader_Bertrand]

Hi WhoKnows, thanks for this detailed post. Have you used TraceOrders to debug the order placing of your strategy better? http://www.ninjatrader-support.com/H...aceOrders.html

Please also post this here, as well as the strategy code you use - if you don't want to attach it here, you can also send it to me to support at ninjatrader dot com Attn Bertrand and I will take care.

Next, this is an important concept to understand - http://www.ninjatrader-support.com/H...tPosition.html

Was your account position flat as you started the strategy? You refer to another position in your last point...I would need to see your full logs from this day to tell you more about this specific one.

I would suggest you clear out this trade and go to Tools > Options > Repair DB and test your strategy again.

Not sure why you see the 'outside trading hours' warning, but are you also monitoring other markets / exchanges that do not fall into the US timezone?

[WhoKnows]

Thanks,

No, I did not try trace orders yet. Will try tomorrow ... now 2AM in Australia... and adapted strategy running (with... if (Historical) return statement; and condition B and D commented out).

RE your answer:
Definitively my strategy and position code are different during my test Replay. Data gathering for this market replay also started after market start.
So.. yes, this might be the reason why the Market position statements didn't work in Replay.

The concept - http://www.ninjatrader-support.com/H...tPosition.html is and was clear to me. Had read that before. That's why I checked positions on Non-EndofDay strategies before starting. In future I will recheck positions after activating the strategies.

Was your account position flat as you started the strategy? Yep, absolutely.
BTW I already Repaired the DB and reset the simulator Replay after each strategy run. So during the Replay the SIM account was also definitively flat.


But independent of this specific issue. Can you pls.try to answers my other questions. I try to grasp what NT does under the hood. Asking you is way faster then trying to find specific replays or simulations.

1.
I think my question boils down to when NT sets the numbers and state of market position? Theoretically; when two conditions are met in the same bar with 'calculate at close of bar=true' in the strategy, can NT then generate two 'sell' orders?
Or to put it in another way: must first receive conformation of a trade before position is set to its new value ... theoretically leaving time for a second order to be issued before the first order is finalized.

2.
The same for 'entries per direction'. When is the flag for the first direction entry set? At the time of the first order, or after conformation from IB, or after the position is filled?

Additionally: If you give two separate orders, 1. going from long to flat and 2. going flat to short. Does this count as 1 entry or 2 entries?

3. See question below. I assume position.quantity is the Strategy position (is not unambiguous in the manual)
And please look at the question... is this the way or is there simpler way to prevent the BarrsArray int to be seen as number of shares?

4. see question. Is there way to compare the Account position with the state of Market position? NT should never allow an Account position 0 to concide with market position=long or short.
Maybe an idea to build this into NT... that at least you will get a warning when this occurs.

5. I think I have only US market orders active. Maybe there are one or two ASX codes... but I am not yet enabled for ASX market data.

6. "You refer to another position in your last point...I would need to see your full logs from this day to tell you more about this specific one."
Ok, will look for the log and send.

[NinjaTrader_JoshP]

1. You can place them immediately provided you have EntriesPerDirection set greater than 1. There is no wait for the first to fill. You submit the first then you immediately submit the second as the code reaches that segment.

2. At the time you submit. There is no check for fill or anything. If the order is submitted and is being processed it will prevent any further orders based on the EntriesPerDirection value.

3. Position.Quantity is strategy position. Everything you do from NinjaScript is always strategy position. I am not sure what you mean by BarsArray int as number of shares. BarsArray is not related with order management.

4. There is currently no way to check anything based off of Accounts. Everything you do is based off of Strategy Positions.

5. I do not believe this should interfere. You strategy might have submitted an order outside of market hours and that is why you got that message.

6. Thank you.

[WhoKnows]

1. Right, but that is not what I asked. Maybe not clear enough. Pls. see the questions in my first posts.
I think I have a strategy that resulted in 2 orders the same direction, in spite of EntriesPerDirection=1. This happened live with FAZ. I still have to analyze how much this issue has cost me.
The questions were designed to help me understand what is happening and how to prevent it. I want to know how to prevent ending up double my strategy account value short. I think it is a bug, but it also could be my poor coding. I adapted this strategy, but I want to understand how to prevent it in future.

BTW This problem does not occur in backtesting, only during market replay and ....live. ....
I could replicate the issue with another instrument on another NT install on another computer.

e.g. Hereby a MarketReplay testrun:
Hour Triggered rules in the strategy Effect(see strategy below)
9.45 1 Entry_L 80 Long SSO ... as set in account value 2000$
13.24 2 & 3 Exit_L 80, Close Position 80, Entry_S 81 -> result 161 Short = double my Strategy Account value
14.36 1 Close Position 161, Entry_L 81
15.09 2 & 3 Exit_L 81, Close Position 81, Entry_S 62 -> result 163 short.... again double exposure
15.24 4 Exit_S 82 -> still 81 left to do further damage
15.39 1 Close Position 81, 81 Long

I sent the relevant files to NT support. Will report back here too.

2. Thank you, .. that is what I understood from the manual. Curious how my strategy manages me to end up double the position anyway.

3. Position.Quantity is the Strategy Position. Thought so, but could not find this stated explicitly.
I was maybe unclear. I meant that if you use, ExitLong(1, "Exit_L", "Entry_L") the strategy will exit with 1 share, while in my strategy that 1 should refer to BarsArray=1... exiting on the added timeframe. So I use now: ExitLong(1, Position.Quantity, "Exit_L", "Entry_L").
Am I correct to use Postion.Quantity here or are there other way to prevent the 1 being seen as 1 share? I did not see this in the manual or examples.

4.

There is currently no way to check anything based off of Accounts. Everything you do is based off of Strategy Positions

.
Too bad as the log file proves that NT sees the Account position. I suggest to add somewhere in your NT coding that if the StrategyPosition is long or short and the the Account Position is 0, that the user gets a warning to check his positions.

5.

You strategy might have submitted an order outside of market hours and that is why you got that message.

Nope, see cvx logfile sent to NT Support ... it happens during the normal trading hours.

6. Log file sent.

[NinjaTrader_JoshP]

1. We cannot assist without logs. With EntriesPerDirection set to 1 the strategy will not make an additional trade. Overfills is a different story and depending on what the logs say we can evaluate if this is the case.

3. That is incorrect. If you use a 3 parameter syntax it is suppose to be quantity. If you wanted BarsArray you need to use the correct syntax for it which is the 4 parameter syntaxing.


ExitLong(int barsInProgressIndex, int quantity, string signalName, string fromEntrySignal)


If you want to use Position.Quantity that is fine. It is dependent on what you want to submit the order as. If you are confused, just store the value you entered at and send your exit long with the same value.


4. Seeing Account position is not possible from the strategy at this point in time. Improvements in this area are already on our list of future considerations.


5. Which email address did you send the logs into with? It would be easier if you could attach them here instead of trying to work through two separate systems. Thank you.

[WhoKnows]

For future reference about the issue of 'overfills' (and prevent other users from incurring potential losses), I thought it would be good to summarize the outcome.


In further testing I could replicate that also with other instruments my strategy was taking double direction entries (resulting in double positions) both in real time and in Market Replay .
Bertrand of NT tested the strategy on his side an could also replicate the problem.
Pls. note that such an issue will not show itself during backtesting !!

Basically the problem was caused by calling ExitLong at the same time as EnterShort. The ExitLong rule did have other period parameters than the EnterShort rule and was added to obtain a fine tuning of exiting the strategy long position before the circumstances were met to enter a short. At times however the (speed of the) price development could trigger both rules in the same bar, triggering both ExitLong and EnterShort at the same time.
Pls. search for 'overfills' to find more about this.

The 'entries per direction' and 'statements to check for Position.Long or Short' were in my strategy not able to prevent overfills.

To quote a follow-up Email of Josh:

Overfills are always a possibility. The Internal Order Handling Rules are in place to reduce the overfill scenarios, but it does not 100% remove them (emphasis added, WhoKnows). Also, since you are using market orders, the rules do not apply. You should become familiar with these risks as they generally govern things like overfills.

I was aware of the text at this link in the manual, but as NT beginner I apparently did/do not yet fully appreciate certain texts in the manual.
IMHO it would improve the NT package/experience substantially if users were more transparently/explicitly warned for certain 'beginners' mistakes.

There are probably more elegant solutions, but for the moment I will remove the use of ExitLong and EnterShort in one strategy, and move to separate Long and Short strategies.

[NinjaTrader_Bertrand]

Hi WhoKnows, you don't have to do a ExitLong and EnterShort on the same bar to reverse, as EnterShort will automatically exit any existing long position and enter a short one.

Please attach the latest revision of your strategy so we can investigate and recommend potential changes.

[WhoKnows]

Hi WhoKnows, you don't have to do a ExitLong and EnterShort on the same bar to reverse, as EnterShort will automatically exit any existing long position and enter a short one.

Thank you, I know, but

The ExitLong rule did have other period parameters than the EnterShort rule and was added to obtain a fine tuning of exiting the strategy long position before the circumstances were met to enter a short.

Sometimes you might want to Exit without Entering the reverse position. I added the extra Exit as it improved my exit efficiency.

But apparently the trigger conditions of only Exit were too close to the trigger conditions of Enter reverse and in a fast moving market both could be triggered within the same bar.
Splitting the old strategy into a separate Long and Short strategy works fine with me...also optimizing and back testing work much faster (on my PC).

[NinjaTrader_Bertrand]

Hi WhoKnows, glad you got it working now.