CashValue and BuyingPower return wrong values

[rmt81]

I am having difficulty running multiple strategies in real time. Each strategy instance has only one instrument.

The basic problem is how to determine when the account has sufficient buying power to place a trade and when it does not. Getting this wrong is disasterous, because if I place an order when the buying power is insufficient, the strategy is cancelled. By the end of a day of trading, I have hardly any strategies still active.

Running in the Sim102 account on live data yesterday, I logged the following sequence of events:
OnStartUp: A, DateTime = 2011-08-18 07:35:35Z, Shares = 0, Price = 0.00, Value = 0, Cash Value = 0, Buying Power = 0
Enter Long: Instr = GME, DateTime = 2011-08-17 15:00:00Z, Shares = 488, Price = 20.46, Value = 9984, Cash Value = 30000, Buying Power = 60000
PartFilled: GME, DateTime = 2011-08-18 08:30:00Z, Shares = 1, Price = 18.71, Value = 19, Cash Value = 20865, Buying Power = 41729
Filled: GME, DateTime = 2011-08-18 08:30:00Z, Shares = 488, Price = 18.72, Value = 9135, Cash Value = 20865, Buying Power = 41729
Enter Long: Instr = CAT, DateTime = 2011-08-18 08:31:00Z, Shares = 118, Price = 84.14, Value = 9929, Cash Value = 20865, Buying Power = 41729
Enter Long: Instr = GIS, DateTime = 2011-08-18 08:31:00Z, Shares = 275, Price = 36.33, Value = 9991, Cash Value = 20865, Buying Power = 41729
Enter Long: Instr = LIFE, DateTime = 2011-08-18 08:31:00Z, Shares = 263, Price = 37.95, Value = 9981, Cash Value = 20865, Buying Power = 41729
Enter Long: Instr = TAP, DateTime = 2011-08-18 08:31:00Z, Shares = 229, Price = 43.53, Value = 9968, Cash Value = 20865, Buying Power = 41729
Enter Long: Instr = JNJ, DateTime = 2011-08-18 08:31:00Z, Shares = 158, Price = 63.21, Value = 9987, Cash Value = 20865, Buying Power = 41729

In the above listing, the lines beginning with “Enter Long:” were generated in OnBarUpdate() immediately following a call to the EnterLong() function and the lines beginning with “PartFilled:” or “Filled:” were generated in OnOrderUpdate().

However, the Orders tab of the Control Center showed the following order history:
Instrument,Time,Action,Order Type,Qty,Limit,Stop,State,Filled,Avg Price,Remaining,Account,Connection
GME,8/18/2011 8:30,Buy,Market,488,0,0,Filled,488,18.71997951,0,S im102 IRA,TD Ameritrade
CAT,8/18/2011 8:31,Buy,Market,118,0,0,Filled,118,84.06974576,0,S im102 IRA,TD Ameritrade
GIS,8/18/2011 8:31,Buy,Market,275,0,0,Filled,275,36.35989091,0,S im102 IRA,TD Ameritrade
LIFE,8/18/2011 8:31,Buy,Market,263,0,0,Rejected: Order has exceeded available account equity: Please reduce order size,0,0,263,Sim102 IRA,TD Ameritrade
TAP,8/18/2011 8:31,Buy,Market,229,0,0,Rejected: Order has exceeded available account equity: Please reduce order size,0,0,229,Sim102 IRA,TD Ameritrade
JNJ,8/18/2011 8:31,Buy,Market,158,0,0,Rejected: Order has exceeded available account equity: Please reduce order size,0,0,158,Sim102 IRA,TD Ameritrade

The implication of this is that the Accounts tab of Control Center showed Cash Value = 945 and Buying Power = 1890 after the first three orders were apparently filled, but GetAccountValue(AccountItem.BuyingPower) returned 20865 as shown above. The strategy does not place orders if BuyingPower < 10500, so the last three orders would not have been placed if GetAccountValue(AccountItem.BuyingPower) returned 1890 as summing the three filled orders indicates it should have been. But, because BuyingPower was not updated correctly, the additional orders were placed, rejected, and terminated the strategies for those instruments for the rest of the day. OnOrderUpdate() seems not to have run either for the orders that were filled, but did not update BuyingPower or for the following three that were rejected, because there should have been lines in the log above corresponding to those executions.I tried to debug this using MarketReplay, but I got a different (correct) result as shown below:Enter Long: Instr = GME, DateTime = 2011-08-17 15:00:00Z, Shares = 488, Price = 20.46, Value = 9984, CV = 20865, BP = 41729
PartFilled: GME, DateTime = 2011-08-18 08:30:00Z, Shares = 1, Price = 18.71, Value = 19, CV = 20865, BP = 41729
Filled: GME, DateTime = 2011-08-18 08:30:00Z, Shares = 488, Price = 18.72, Value = 9135, CV = 20865, BP = 41729
Enter Long: Instr = GIS, DateTime = 2011-08-18 08:31:00Z, Shares = 275, Price = 36.33, Value = 9991, CV = 10866, BP = 21731
PartFilled: GIS, DateTime = 2011-08-18 08:31:00Z, Shares = 3, Price = 36.35, Value = 109, CV = 10866, BP = 21731
Filled: GIS, DateTime = 2011-08-18 08:31:00Z, Shares = 275, Price = 36.36, Value = 9999, CV = 10866, BP = 21731
Enter Long: Instr = CAT, DateTime = 2011-08-18 08:31:00Z, Shares = 118, Price = 84.30, Value = 9947, CV = 937, BP = 1874
PartFilled: CAT, DateTime = 2011-08-18 08:31:01Z, Shares = 4, Price = 84.13, Value = 337, CV = 937, BP = 1874
Filled: CAT, DateTime = 2011-08-18 08:31:01Z, Shares = 118, Price = 84.14, Value = 9928, CV = 937, BP = 1874

Note that the execution order is different than above. I initially ran the strategy for all six stocks but LIFE and TAP filled after GME, so I reran it with only GME, CAT, and GIS to obtain the result above. I assume this is due to rounding the times in the Market Replay data. (A sequential number stored along with the time to maintain the order would be really useful for debugging cases like this would be very helpful.)
I suspect that in the real-time data, GIS and CAT may have filled (or partially filled) in between the partial fill of 1 share of GME and the additional 487 shares that filled the order. But, that does not explain why the orders for GIS and CAT filled in the actual account in Control Center, but did not trigger OnOrderUpdate() or update CashValue and BuyingPower.

I don’t know how to debug this further if I have to be in real time to see the problem. I can’t trap for the error if I don’t know when it is coming.

Any suggestions you can provide to explain this would be greatly appreciated.

Thanks

[NinjaTrader_Brett]

Hello,

Wanted to let you know I received this.

I will respond here in a few hours. Thanks for your patience.

[NinjaTrader_Brett]

Hello,

Thanks for your patience.

Just want to make sure I understand this.

So the only problem scenario here is when the trades are places at the same exact time correct?

Thanks.

[rmt81]

CashValue and BuyingPower return wrong values

Hello,

Thanks for your patience.

Just want to make sure I understand this.

So the only problem scenario here is when the trades are places at the same exact time correct?

Thanks.

Brett:


I don't know whether multiple trades have to be simultaneous or not. That is one possibility. This effect seems diffiicult to replicate.


The most consistent factor seems to be that it occurs when trading live in the simulated account. I think it also happened when I tried trading live on my real account, although I failed to save an output log of that trading session.


I have not been able to replicate this effect in backtesting or when running on MarketReplay data.


So, I seem to be able to rely on the CashValue reported (provided I place only long positions) if I am running MarketReplay, but when I am live, some transactions do not update CashValue and before long I issue a buy with insufficient funds and then the whole strategy soon crumbles.


The reason for having multiple buys all at one time seems to be that when the strategy starts running, the indicator is not based on sufficient historical data to stabilize the moving averages, so a substantial fraction of the stocks I am running the strategy on issue buy signals in the first bar. This is a secondary problem that may exacerbate the first problem, but I don't think that fixing the second problem will fix the first, although it might make it less likely.


The second problem (jumping into many stocks at the beginning of the trading session) does not occur in backtesting, but does seem to occur in MarketReply, live simulation, and live real trading.


I will need to solve both of these problems before I can try trading real money again.


Thanks

[NinjaTrader_Brett]

Hello,

I believe this is the primary issue you are seeing.

If multiple strategies takes trades at the same time this will cause the issue.

The CashBalance will not change until after the trade is submitted and filled.

It takes milliseconds after the order is submitted to fill the order. By then the other strategies have already run there own strategies and tried to submit their own trades.

This would be expected. The only way you would be able to work around this problem is communication between strategies. So that when one strategies tried to take a position the others do not until it is filled and the cash value is updated. This would require a flag that can be attempted with a public variable that is visable to all strategies.

This is 100% unsupported however but would be the only way I can think of to solve this.

Let me know if I can be of further assistance.

[rmt81]

Hello,

I believe this is the primary issue you are seeing.

If multiple strategies takes trades at the same time this will cause the issue.

The CashBalance will not change until after the trade is submitted and filled.

It takes milliseconds after the order is submitted to fill the order. By then the other strategies have already run there own strategies and tried to submit their own trades.

This would be expected. The only way you would be able to work around this problem is communication between strategies. So that when one strategies tried to take a position the others do not until it is filled and the cash value is updated. This would require a flag that can be attempted with a public variable that is visable to all strategies.

This is 100% unsupported however but would be the only way I can think of to solve this.

Let me know if I can be of further assistance.

Brett:

Thanks, I think that addresses the issue I am seeing here, although it is still difficult to replicate.

Does NT include anything like the concept of locking a record in a database? What I am suggesting is that, if the user requests to lock the account, between the time that an order is placed and when it is either filled or rejected, the account is locked so that no other orders can be placed against it.

Where in the code could I define a variable that would be accessible to other strategies?

Thanks

[NinjaTrader_Brett]

Hello,

No problem.

Unfortunately there is nothing provided from NT that is supported around this. There is some unsupported coding you could do around declaring a public variable that is the accessible from all strategies.

Let me know if I can be of further assistance.