MarketDepthEventArgs.Price is always set to zero for row removal operations

[ScoobyStoo]

Hi guys,

The MarketDepthEventArgs.Price is always set to zero when MarketDepthEventArgs.Operation == Operation.Remove.

e.g.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 5
Price : 0
Volume : 0

Could you please fix it up so that the price is populated as it was in NT6.5.

Thanks

[NinjaTrader_Dierk]

>> The MarketDepthEventArgs.Price is always set to zero when MarketDepthEventArgs.Operation == Operation.Remove.
Correct, that's how it's supposed to work as price and volume are irrelevant on removal. There is nothing to "fix".

[ScoobyStoo]

>> The MarketDepthEventArgs.Price is always set to zero when MarketDepthEventArgs.Operation == Operation.Remove.
Correct, that's how it's supposed to work as price and volume are irrelevant on removal. There is nothing to "fix".

OK, right. So it's a breaking change between NT6.5 and NT7. I'll alter my code accordingly.

[NinjaTrader_Dierk]

Incorrect, Price and Volume never should be been used on Operation=Remove events. Please use .Position to properly process that event. Thanks

[ScoobyStoo]

Incorrect, Price and Volume never should be been used on Operation=Remove events. Please use .Position to properly process that event. Thanks

Dierk,

This functionality was exposed through the NT6.5 API and is no longer exposed through the NT7 API. Therefore by definition it is a breaking change. Please do not presume to know how your customers are using the API.

Thanks

[NinjaTrader_Dierk]

Using Price/Volume for removal ALWAYS was incorrect. Please let me know if you found anything different in our samples or documention so it could be addressed. For that reason this is NOT a "code breaking change".

[ScoobyStoo]

Using Price/Volume for removal ALWAYS was incorrect. Please let me know if you found anything different in our samples or documention so it could be addressed. For that reason this is NOT a "code breaking change".

You might want to change these pages in your NT6.5 documentation then which make no mention of this fact.

http://www.ninjatrader-support.com/HelpGuideV6/OnMarketDepth.html

http://www.ninjatrader-support.com/HelpGuideV6/MarketDepthEventArgs.html

Can I ask why it was removed? It's not a big deal, but it saved me a few lines of code not having to query my custom order book to establish the price of the removed row.

[NinjaTrader_Dierk]

Unfortunately we do not provide insight on why code which is used against what we document or provide samples on is changed.

Please see here for how to correctly build an order book: http://www.ninjatrader-support2.com/...ead.php?t=3478

[ScoobyStoo]

Unfortunately we do not provide insight on why code which is used against what we document or provide samples on is changed.

Please see here for how to correctly build an order book: http://www.ninjatrader-support2.com/...ead.php?t=3478

Thanks Dierk,

I hadn't seen that example before. Having scanned over the sample code though I can see that it is likely to break when using the downloadable NT7 replay data because the order book is not properly initialised to reflect the current order book state (i.e. all rows inserted) at the start of a replay session.

e.g.

The line...

if (e.Operation == Operation.Insert)[INDENT]rows.Insert(e.Position, new LadderRow(e.Price, e.Volume, e.MarketMaker));
[/INDENT]

will throw an ArgumentOutOfRangeException if the first market depth event the order book encounters in a replay session happens to be an insert at position 5.

I think this is down to the fact that the downloadable data comes from a continuous recording which is split into separate files. However, when a customer records their own replay files they don't experience this as the market data providers tend to send a set of order book initialisation L2 messages when the subscription to the instrument is created.

From my experience the NT6.5 market replay engine is pretty clever in that if you jump around in the timeline the first L2 events always initialise the order book using a set of insert operations to populate the order book. However, obviously this relies on order book data being available in the replay file for the period before the point in the timeline where you are about to start replaying. Looks like NT7 works the same way so a solution could be to always shift slightly forward in the timeline before starting to replay.

Anyways, I have my own workaround for this now so thanks for your help on this issue.

[ScoobyStoo]

Actually, I take that back. The workaround won't work because strategies are now disabled whilst you jump to another point in the market replay timeline (see event dump below). This means that the order book population events are raised whilst the strategy is disabled. This is a shame. It was a useful feature in NT6.5.

**NT** Disabling NinjaScript strategy 'TestStrategy/dfde4c6a64384cdcaa639a93914d185f'
Level 2 Market Data
Time : 01:00:34.968
Type : Ask
Operation : Insert
Position : 0
Price : 1.387
Volume : 12


Level 2 Market Data
Time : 01:00:31.968
Type : Ask
Operation : Insert
Position : 1
Price : 1.3871
Volume : 49


Level 2 Market Data
Time : 01:00:35.968
Type : Ask
Operation : Insert
Position : 2
Price : 1.3872
Volume : 40


Level 2 Market Data
Time : 01:00:30.968
Type : Ask
Operation : Insert
Position : 3
Price : 1.3873
Volume : 51


Level 2 Market Data
Time : 01:00:35.968
Type : Ask
Operation : Insert
Position : 4
Price : 1.3874
Volume : 24


Level 2 Market Data
Time : 01:00:34.968
Type : Bid
Operation : Insert
Position : 0
Price : 1.3868
Volume : 22


Level 2 Market Data
Time : 01:00:34.968
Type : Bid
Operation : Insert
Position : 1
Price : 1.3867
Volume : 65


Level 2 Market Data
Time : 01:00:34.968
Type : Bid
Operation : Insert
Position : 2
Price : 1.3866
Volume : 55


Level 2 Market Data
Time : 01:00:34.968
Type : Bid
Operation : Insert
Position : 3
Price : 1.3865
Volume : 94


Level 2 Market Data
Time : 01:00:35.968
Type : Bid
Operation : Insert
Position : 4
Price : 1.3864
Volume : 37


**NT** Enabling NinjaScript strategy 'TestStrategydfde4c6a64384cdcaa639a93914d185f' : On starting a real-time strategy - StrategySync=WaitUntilFlat SyncAccountPosition=False EntryHandling=AllEntries EntriesPerDirection=1 StopTargetHandling=PerEntryExecution ErrorHandling=TakeNoAction ExitOnClose=True/ triggering 30 before close Set order quantity by=Strategy ConnectionLossHandling=StopStrategy DisconnectDelaySeconds=10 CancelEntryOrdersOnDisable=False CancelExitOrdersOnDisable=False MaxRestarts=4 in 5 minutes

[NinjaTrader_JoshP]

The line...

Code:
if (e.Operation == Operation.Insert)

rows.Insert(e.Position, new LadderRow(e.Price, e.Volume, e.MarketMaker));

will throw an ArgumentOutOfRangeException if the first market depth event the order book encounters in a replay session happens to be an insert at position 5.

Do you have a specific replay file where this is the case?

[ScoobyStoo]

Do you have a specific replay file where this is the case?

Nope, afraid not. Was just working through the code's logic. But if the order book is not sequencially initialised (i.e. position 0 insert, position 1 insert, ...) you can bet it'll crop up at some point. One way around this is to populate any empty row positions with nulls and then properly populate them later via 'update' operations.

The other problem with this code is that if the order book is not initialised then it will take an age to become fully populated because the 'update' and 'remove' operations will not populate the rows unless those rows have been inserted via an 'insert' operation.

IMHO this code doesn't work unless the order book is initialised correctly. It is much easier to implement an order book using a SortedList with the row price as the key. Order book maintenance becomes a cinche then and it's actually faster than the method shown here. Unfortunately, this method cannot be used so easily anymore because the price property is not populated for 'remove' operations.

Any chance you could reinstate it?

[ScoobyStoo]

Hi Josh,

There seems to be a problem with the allocation of the position indexes anyay. Please take a look at the L2 market data event dump below and I'll explain. This was obtained using NT's downloadable data for the 6E 03-10 on Sunday Jan 31st so you should be able to replicate it yourself.

Market opens

Level 2 Market Data
Time : 23:00:00.968
Type : Bid
Operation : Update
Position : 0
Price : 1.3859
Volume : 6


Level 2 Market Data
Time : 23:00:00.968
Type : Ask
Operation : Update
Position : 0
Price : 1.386
Volume : 9


Level 2 Market Data
Time : 23:00:00.968
Type : Ask
Operation : Update
Position : 2
Price : 1.3864
Volume : 10


Level 2 Market Data
Time : 23:00:00.968
Type : Ask
Operation : Update
Position : 0
Price : 1.386
Volume : 8


Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Update
Position : 0
Price : 1.386
Volume : 7


Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 4
Price : 1.3866
Volume : 1


Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 5
Price : 0
Volume : 0


Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Update
Position : 4
Price : 1.3866
Volume : 2

Notice that the row with price 1.3866 is at position 4.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 0
Price : 0
Volume : 0

The row at position 0 is removed meaning that the row with price 1.3866 drops to position 3.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 5
Price : 1.3867
Volume : 18

A row with price 1.3867 is inserted at position 5. However, this breaks the position indexing as the row at position 4 must have a price between 1.3866 and 1.3867, which is impossible.

Level 2 Market Data
Time : 23:00:03.968
Type : Ask
Operation : Update
Position : 0
Price : 1.3863
Volume : 2


Level 2 Market Data
Time : 23:00:03.968
Type : Bid
Operation : Insert
Position : 0
Price : 1.386
Volume : 1


Level 2 Market Data
Time : 23:00:03.968
Type : Bid
Operation : Remove
Position : 5
Price : 0

Can I make an empassioned plea (this comes from a man slowly losing his sanity trying to get market replay working properly) that you reinstate the price property for 'remove' operations and let those of us using the NT API to manage our own custom order books do it using just the price and quantity data rather than forcing us to use the position property which doesn't work.

Many thanks.

[NinjaTrader_JoshP]

ScoobyStoo,

In our previous exchanges we concluded that data before February should not be used as examples as data server changes occurred afterwards. Please test your scenarios with data after then to see the behavior.

Also, can you please attach your dump strategy. Thank you.

[ScoobyStoo]

Josh,

The data I tested related to the first week of Feb 2010 (the week happened to start on the evening of Sun 31st Jan). The data dump strategy simply calls ToString() on the properties of the MarketDataEventArgs object.

Sorry, I don't have any more spare time to spend on this issue. The data integrity is good, it's the position assignment logic that looks to be flawed. Either the price property needs to be reinstated to allow your customers to manage their own order books or the position logic needs to be fixed.

Best of luck.