MarketDepthEventArgs.Price is always set to zero for row removal operations

[ScoobyStoo]

No exception will be thrown. If the position given is greater than anything on the book it is simply ignored.

Even worse! The row is left in the book. So what happens if the event sequence is as below (which is very common when quotes are flashed):

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 5
Price : 1.3867
Volume : 1

This row actually gets inserted into the custom book at position 4 using the new sample code.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 5
Price : 0
Volume : 0

There is no row at position 5 to be removed and so the row at position 4 is left in the order book.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 5
Price : 1.3867
Volume : 1

A row is reinserted into position 5 leading to the custom order book containing 2 rows with price = 1.3867.

You need to manage the positions through code because when you are using OnMarketDepth() you are literally accessing the raw underlying data from your data provider. This is how they provide it to us and that is how it is provided to you.

Really? Then why isn't the price information published for removal operations? Every data provider I have used publishes this information for every order book operation.

[NinjaTrader_JoshP]

Not sure what you are finding as the issue. When there is no position 5, there is nothing to remove and nothing will be removed. When a new insert happens at position 5 you will be inserted correctly into position 5. I suggest you follow the actual prices between each row to see that all the rows line up accurately.

I am not sure how you get prints that do not show prices for removal operations. Please follow the reference sample and just print out e.ToString(). You will be able to find the prices for the various rows.

[ScoobyStoo]

Not sure what you are finding as the issue. When there is no position 5, there is nothing to remove and nothing will be removed. When a new insert happens at position 5 you will be inserted correctly into position 5. I suggest you follow the actual prices between each row to see that all the rows line up accurately.

But that's my point. By working through the logic it's possible to deduce that the prices won't line up accurately in some cases. In the example I just provided you end up with 2 rows with the same price.

Anyhow, unfortunately I really can't spend anymore time on this. I'll plug in the sample code you provided and add some code to detect any impossible row position assignments such the ones we've already discussed in this thread. If/when I find one I'll drop you a mail with the details so you can verify it at your end.

I am not sure how you get prints that do not show prices for removal operations. Please follow the reference sample and just print out e.ToString(). You will be able to find the prices for the various rows.

This works fine for live data but all the downloadable Market Replay data (which is what I'm interested in) has price == 0 for removal operations. Can't use my personal MR data which I recorded using NT6.5 because of issues with replaying this data in NT7.

[NinjaTrader_JoshP]

ScoobyStoo,

Unfortunately I am not sure, in all the replay files I downloaded and tested on NT7, removal operations all come with prices.

[NinjaTrader_Dierk]

Josh's updated sample matches NT internal logic 100%. This logic works since day 1 of NT. As you can see there absolutely NO need for having a price on a removal operation. I suggest sticking to the EXACT logic as outlined by Josh's sample in order to avoid any further confusion.

Thanks

[ScoobyStoo]

ScoobyStoo,

Unfortunately I am not sure, in all the replay files I downloaded and tested on NT7, removal operations all come with prices.

Josh,

I downloaded the ESM0 data for yesterday and this does indeed have the price property correctly populated for removal operations. To whoever reinstated this I offer my genuine thanks. However, can you let me know when this change to your market data recording logic was made, as the 6E data I have downloaded for the month of Feb 2010 has the price set to zero. Obviously I'd like to avoid having to try to establish by trial and error the date when the price started being populated.

Thanks.

[NinjaTrader_Dierk]

Nothing has changed with the way on how L2 data recorded. Again, I ask you to strictly follow Josh's sample in order to avoid an further confusion. This sample clearly points out that .Price is NOT needed for removal. I makes NO sense to base your logic on anything different. Thanks for your understanding.

[ScoobyStoo]

Nothing has changed with the way on how L2 data recorded. Again, I ask you to strictly follow Josh's sample in order to avoid an further confusion. This sample clearly points out that .Price is NOT needed for removal. I makes NO sense to base your logic on anything different. Thanks for your understanding.

Dierk,

Well, looks like we are destined to eternally disagree on this one and there's no point in going over the same old ground again.

Interestingly, it looks as if the price property is populated for the ES contract but not the 6E contract which is a bit bizarre.

Anyway, time to move on to pastures new.

Thanks for your help with this issue.

[ScoobyStoo]

Dierk/Josh/Bertrand,

I need to reopen this thread as I have come back to NinjaTrader after a couple of months working on something else and am experiencing problems with your downloadable market replay data.

Please load the sample indicator that Josh supplied and then replay any of the downloadable 6E 03-10 data for Feb 2010. I have tried multiple dates and am seeing the following exception being thrown at some point in every session.

You are accessing an index with a value that is invalid since its out of range.

Please confirm whether you see the same behaviour.

Thanks

[NinjaTrader_Dierk]

Please let us know as you feel the updated sample provided by Josh would not work on that data.

[ScoobyStoo]

Hi Dierk,

OK, here's a random example. Download the 6E 03-10 data for 21st Feb. Attach that sample indicator to a chart and start replaying. An exception is immediately thrown at the line

rows[e.Position].MarketMaker = e.MarketMaker;

because no row with the stated position exists.

The code assumes that any update operations refer to row positions that have already been added which is not the case in the replay data.

Thanks

[NinjaTrader_Dierk]

The NT7 (!) sample provided here

http://www.ninjatrader.com/support/forum/showthread.php?t=3478

... works no problem using replay data for 6E 03-10 for 21th of Feb 2010 on a 1 minute chart: no error in the logs.

Same for L2 window for 6E 03-10: all works as expected.

[ScoobyStoo]

That is the sample code I am using.

I am using NT7.0.0.16 and my chart is the 6E 03-10 (1 Min) 21/02/2010.

This is the output:

Ask Book
Bid Book
Ask Book
Bid Book
Ask Book
Bid Book
Error on calling 'OnMarketDepth' method for indicator 'SampleMarketDepth': You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.

[ScoobyStoo]

I have found the source of the problem.

The market depth events are not being fired until the required number of bars have elapsed. Since the BarsRequired property defaults to a value of 20 none of the market depth events up to bar 20 are being fired. This results in the loss of market depth messages.

The BarsRequired property must be set to 0 to ensure the sample code works.

This is not the behaviour described in the documentation which states:

"Definition
The number of bars on a chart required before the indicator plots. This is NOT the same as a minimum number of bars required to calculate the indicators values."

http://www.ninjatrader.com/support/helpGuides/nt7/barsrequired.htm

I believe this is a bug in the way BarsRequired is implemented in NT7.0.0.16.

[NinjaTrader_JoshP]

ScoobyStoo,

To clarify, you are using the NT7 version listed in that reference sample section correct? I am not seeing any issues on my end like Dierk either. Do you have any special settings for it being used? Maybe special chart settings? Session template? Etc. Thanks.

Also, for the BarsRequired one, we will get back to you at a later point in time.