MarketDepthEventArgs.Price is always set to zero for row removal operations

[NinjaTrader_JoshP]

ScoobyStoo,

Thank you for your efforts, but can you please attach the exact strategy you are using so there will be no ambiguity into what exactly is being used? We can take it from there afterwards. Thanks.

[ScoobyStoo]

ScoobyStoo,

Thank you for your efforts, but can you please attach the exact strategy you are using so there will be no ambiguity into what exactly is being used? We can take it from there afterwards. Thanks.

No problem Josh, see attached file.

Please let me know when you come up with a solution.

All the best

[NinjaTrader_JoshP]

ScoobyStoo,

It appears your sample doesn't actually keep track of the bid/ask books. In a few moments here please find my attached sample and output.

[ScoobyStoo]

ScoobyStoo,

It appears your sample doesn't actually keep track of the bid/ask books. In a few moments here please find my attached sample and output.

Hi Josh,

Happy to have a look at any sample code but I don't see how tracking ask/bid rows has any impact on the problem of the row position being calculated incorrectly by NT.

In order to manage a custom order book I either need to know either:

(a) the price

or

(b) the row index

of the insert/update/remove operations.

I can't use the price because it is no longer published for removal operations and I can't reliably use the row index because NT seems to exhibit a few problems when it generates it.

If either the price or correct row index is provided for each update then maintaining the book becomes a fairly trivial process.

[NinjaTrader_JoshP]

ScoobyStoo,

Please disregard my earlier post at this point in time. We are looking into something. Thanks for your patience.

[NinjaTrader_Dierk]

ScoobyStoo, your analysis helped us to understand the issue which will result in updated sample. Basically there are some bounds checking missing which NT internally has in place since day 1 and which make sure NT internal order book keeping e.g. for the L2 would stay in sync.

Thanks for your patience on that matter. Please allow Josh some time to update the sample and then amend your code accordingly. You'll find that the .Position property is the relevant property for building the book.

[MXASJ]

This may be a totaly wrong way to do it but I've been looking at a way to check the "real" spread between two instruments, where a certain depth is required before the "real" spread is calculated (a realAsk-realBid calc). The code seems to print what I want it to print, but is not 100% tested . I'm keeping an eye out for a better way to do this.

protected override void OnMarketDepth(MarketDepthEventArgs e)///////////////////////////////////////////////////////
{
 
try
{
if (BarsInProgress == 0) // The Quoted instrument 
{
if (e.MarketDataType == MarketDataType.Bid && e.Position == 0 && e.Volume >= bidDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL BID is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realBid.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Bid && e.Position == 1 && e.Volume >= bidDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL BID is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realBid.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Bid && e.Position == 2 && e.Volume >= bidDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL BID is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realBid.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Bid && e.Position == 3 && e.Volume >= bidDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL BID is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realBid.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Bid && e.Position == 4 && e.Volume >= bidDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL BID is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realBid.Set(e.Price);
}
}
}
catch (Exception)
{
Log("TryCatch Error: STRATEGY NinjaSpreader EOD3. OnMarketDepth/Quoted Instrument.", LogLevel.Error);
Print(Time[0] + " " + e.ToString());
}
try
{ 
if (BarsInProgress == 1) //The Hedge instrument
{
if (e.MarketDataType == MarketDataType.Ask && e.Position == 0 && e.Volume >= askDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL ASK is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realAsk.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Ask && e.Position == 1 && e.Volume >= askDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL ASK is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realAsk.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Ask && e.Position == 2 && e.Volume >= askDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL ASK is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realAsk.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Ask && e.Position == 3 && e.Volume >= askDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL ASK is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realAsk.Set(e.Price);
}
else if (e.MarketDataType == MarketDataType.Ask && e.Position == 4 && e.Volume >= askDepth && e.Operation == Operation.Update)
{
if (printDepth) Print("STRATEGY NinjaSpreader "+ Instrument.FullName+" REAL ASK is " + e.Price + " " + e.Position+ " VOL " + e.Volume);
realAsk.Set(e.Price);
}
}
}
catch (Exception)
{
Log("TryCatch Error: STRATEGY NinjaSpreader EOD3. OnMarketDepth/Hedge Instrument.", LogLevel.Error);
Print(Time[0] + " " + e.ToString());
}
if (BarsInProgress == 0 || BarsInProgress == 1)
{
realSpread.Set(realAsk[0]-realBid[0]);
if (printRealSpread) PrintWithTimeStamp("Real Spread is: " +realSpread);
}
} 

[NinjaTrader_JoshP]

ScoobyStoo,

Please find the attached sample.

Basically, you cannot simply rely on e.Position. You actually have to build a book to determine the correct row position the event should allocate. This is the case because you can receive e.Position values that are greater than the current size of your book. Please see the sample which uses the same logic as our internal NT L2 book.

[MXASJ]

Josh I can open and view that zip file, but not Import it. The NT error I get is "Version required to extract this entry not supported (45)". No big as I can copy-paste, but FYI...

NT7b13, Win7Pro/32

[NinjaTrader_JoshP]

Thanks. This may have occurred because I exported from a custom version of NT7.

[ScoobyStoo]

ScoobyStoo,

Please find the attached sample.

Basically, you cannot simply rely on e.Position. You actually have to build a book to determine the correct row position the event should allocate. This is the case because you can receive e.Position values that are greater than the current size of your book. Please see the sample which uses the same logic as our internal NT L2 book.

Thanks Josh.

Can't seem to unzip the file though. Looks to be corrupt.

Can you repost please?

[NinjaTrader_JoshP]

ScoobyStoo,

I reposted it in my earlier post. Can you please see if that one works? Thanks.

[ScoobyStoo]

ScoobyStoo,

I reposted it in my earlier post. Can you please see if that one works? Thanks.

Thanks Josh.

This one unzips with no problems.

Will take a look at it over the weekend.

[ScoobyStoo]

Josh,

This new sample looks to function in a similar way as the sample Dierk pointed me towards.

The key lines of code are:

            // Checks to see if the action taken was an insertion into the ladder
            if (e.Operation == Operation.Insert)
            {
                // Add a new row at the end if the designated position is greater than our current ladder size
                if (e.Position >= rows.Count)
                    rows.Add(new LadderRow(e.Price, e.Volume, e.MarketMaker));
                
                // Insert a new row into our ladder at the designated position
                else
                    rows.Insert(e.Position, new LadderRow(e.Price, e.Volume, e.MarketMaker));
            }
            
            /* Checks to see if the action taken was a removal of itself from the ladder
            Note: Due to the multi threaded architecture of the NT core, race conditions could occur
            -> check if e.Position is within valid range */
            else if (e.Operation == Operation.Remove && e.Position < rows.Count)
                rows.RemoveAt(e.Position);

It is now trapping for e.Position >= rows.Count on insertions but (with reference to the market data event sequence in the previous post #13 on this thread) what would happen if this sequence continued as below.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Update
Position : 4
Price : 1.3866
Volume : 2

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 0
Price : 0
Volume : 0

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 5
Price : 1.3867
Volume : 18

This row actually gets inserted into the custom book at position 4 using the new sample code.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Remove
Position : 5
Price : 0
Volume : 0

There is no row at position 5 to be removed and an exception will be thrown using the new sample code.

Level 2 Market Data
Time : 23:00:01.968
Type : Ask
Operation : Insert
Position : 0
Price : 1.385
Volume : 7

You see my point? Putting a cludge in isn't an elegant or robust solution. We need to address the real problem here, which is either:

(1) The position calculation logic being incorrect.

Why does the new sample code need to reposition the row? Why doesn't your internal logic which calculates the position index assign rows the correct position?

(2) The market data events being raised out of sequence.

The position indexing in #13 is correct if the insertion at position 5 occurs before the removal from position 0. Is there any reason why these might be raised out of sequence (threading issues etc)?

Sorry guys, I really don't know how many other ways to say this. The position index isn't required to manage a custom order book and just adds an unnecessary level of complication. The .NET BCL provides the SortedList class for exactly these kind of usage scenarios. You just need one SortedList for the ask rows and another SortedList (constructed using an inverse comparer) for the bid rows. Then everything can be easily and safely managed using just price and size data. All the data can still be accessed by index if required (except the index is accurately maintained by the SortedList). This is precisely why protocols such as FIX don't need to expose a position field for market data messages.

I'm not asking you to change your API as obviously a lot of your customers will be using the position index and need to continue doing so. However, I haven't been given a good reason why you have stopped publishing the price data for removal operations. It is perfectly valid data and should be published.

Please give it some serious consideration. If there wasn't a good reason to remove it then perhaps it can be reinstated in the next beta release?

Thanks.

[NinjaTrader_JoshP]

No exception will be thrown.

else if (e.Operation == Operation.Remove && [B]e.Position < rows.Count[/B])
     rows.RemoveAt(e.Position);

If the position given is greater than anything on the book it is simply ignored. You need to manage the positions through code because when you are using OnMarketDepth() you are literally accessing the raw underlying data from your data provider. This is how they provide it to us and that is how it is provided to you. To actually have a book, you will need to apply the same logic as NT does for its own books.