Syncing OnMarketData with OnMarketDepth

[iantg]

Hi,

I am trying to build a data model where I can see the following for a new price level created in the inside bid / ask once the price moves into the new price level:

1. Starting volume
2. Total Transacted volume
3. Ending Volume

I can pick up the starting volume and ending volume from OnMarketDepth with something like.

if (marketDepthUpdate.MarketDataType == MarketDataType.Ask &&( marketDepthUpdate.Operation == Operation.Update || marketDepthUpdate.Operation == Operation.Add)&& GetCurrentAsk() == marketDepthUpdate.Price)

Then I can filter for price level changes by setting a variable to the old GetCurrentAsk() and when the new GetCurrentAsk() != to my variable I know that the price level has changed.

For the transacted volume: I can run something like this from OnMarketData:

if (marketDataUpdate.MarketDataType == MarketDataType.Last)
{
if (marketDataUpdate.Price >= marketDataUpdate.Ask)
{
Print( "Transacted Volume" + marketDataUpdate.Volume);

}

This is just some flavor of: https://ninjatrader.com/support/help...marketdata.htm

Both of these methods work fairly well independently and give me the data that I need. However they are not in sync together as far as the timing. For example, when the OnMarketDepth event handler is switching from price level x to price level y, the OnMarketData event handler is more often than not lagging and still stuck on price level x.

So getting these two to sync up is a bit of a challenge. I am sure I am not the first guy out there trying to get information from both event handlers at the same time concerning the same price level event. After searching this forum, I haven't found any information on this topic, so I figured I would post the question.

Can someone from NT point me toward the best practice for how someone can get data for a single bid or ask price from these two event handlers in sync?

Thanks,

Ian

[NinjaTrader_Jim]

Hello iantg,

Thanks for your question.

I'll work on a demonstration script for this task, and I'll update this post when it is prepared.

Please allow a few days for me to work on this item. Thanks in advance for your patience.

[iantg]

Hi Jim,

Thanks for offering to help me on this. I have built a few methods that kind of sort of work, but I just keep getting out of sequence somehow, so any help you can provide would be greatly appreciated.

Thanks,

Ian

Hello iantg,

Thanks for your question.

I'll work on a demonstration script for this task, and I'll update this post when it is prepared.

Please allow a few days for me to work on this item. Thanks in advance for your patience.

[NinjaTrader_Jim]

Hello iantg,

I have had a discussion with Product Management on this particular item. They had informed me that there is not a guarantee of the sequence of events between OnMarketData and OnMarketDepth, so you will want to use one or the other depending on your requirements.

If there is another matter you would like us to look into, I'll be more than happy to assist you further.

[zundradaniel]

Sorry to dig up a 6 year old post NinjaTrader_Jim , but I'm working on something that has a similar requirement. I wasn't looking for a solution to the sync, but rather ran across this while looking up documention on the OnMarketDepth method.

However, before I get too far into this thing, I had an assumption that the order of events could be guaranteed just by virtue of how the market functions vs a technical thread safe guarantee. For example, a market order cannot hit the orderbook unless there are orders present. So it follows that OnMarketDepth must preceed OnMarketData events right? The problem I can see here is the timing of the operation when indicator first loads (at the milisecond when it starts, what was happening, i.e. did a market order fire before the OnMarketDepth was fully initialized). I think this could be handled by guarding the call to the datastructure holding the DOM data to ensure the key is present before running any calculations. This initialization edge case should only happen the first time a market order is sent prior to the DOM being loaded into memory.

Also, for thread saftey purposes, we can also use a thread lock on a global data structure that is responsible for storing both the DOM data and the volume data between the two functions.

Would this work or am I missing something am about to waste a bunch of time trying to get something to work that has a critical flaw?

[iantg]

You are wasting your time. OnBarUpdate front runs all other methods by anywhere from a few hundred milliseconds to a few seconds. There is no way to change this. For a variety of reasons NinjaTrader will always be too slow to pursue any microstructure based edges, HFT or even scalping based on level 2 data at a granular level IMO. It’s great for longer timeframes, but if you need more speed, control, or want to purse HFT you should consider building your own tech stack from scratch. Nothing, and I mean literally nothing in retail will even get you close to what you need.

[zundradaniel]

You are wasting your time. OnBarUpdate front runs all other methods by anywhere from a few hundred milliseconds to a few seconds. There is no way to change this. For a variety of reasons NinjaTrader will always be too slow to pursue any microstructure based edges, HFT or even scalping based on level 2 data at a granular level IMO. It’s great for longer timeframes, but if you need more speed, control, or want to purse HFT you should consider building your own tech stack from scratch. Nothing, and I mean literally nothing in retail will even get you close to what you need.

The sync wouldn't happen in OnBarUpdate (it couldn't since this is yet a 3rd thread and locking state there wouldn't do anything to control flow between the other two in OnMarketUpdate and OnMarketDepth). I've done some further testing and this works fine:

namespace NinjaTrader.NinjaScript.Indicators
{
    public class TestDepthVolumeSync : Indicator
    {
        private SortedList<double, DomRow> domRows;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description = "Enter the description for your new custom Indicator here.";
                Name = "TestDepthVolumeSync";
                Calculate = Calculate.OnEachTick;
                IsOverlay = false;
                DisplayInDataBox = true;
                DrawOnPricePanel = true;
                DrawHorizontalGridLines = true;
                DrawVerticalGridLines = true;
                PaintPriceMarkers = true;
                ScaleJustification = NinjaTrader.Gui.Chart.ScaleJustification.Right;

                // Disable this property if your indicator requires custom values
                // that cumulate with each new market data event.
                IsSuspendedWhileInactive = true;
            }
            else if (State == State.Configure)
            {
                domRows = new SortedList<double, DomRow>();
            }
        }

        protected override void OnMarketDepth(MarketDepthEventArgs e)
        {
            lock (domRows)
            {
                switch (e.Operation)
                {
                    case Operation.Add:
                    case Operation.Update:
                        if (!domRows.ContainsKey(e.Price))
                            domRows[e.Price] = new DomRow(ConvertVolume(e.Volume), e.MarketDataType, e.Price, e.Position);
                        else
                            domRows[e.Price].SetData(ConvertVolume(e.Volume), e.Position);
                        break;

                    case Operation.Remove:
                        domRows.Remove(e.Price);
                        break;
                }
            }
        }

        protected override void OnMarketData(MarketDataEventArgs m)
        {
            lock (domRows)
            {
                if (m.MarketDataType == MarketDataType.Last)
                {
                    if (!domRows.ContainsKey(m.Price))
                    {
                        Print($"Price {m.Price} not found in DomRows");
                        return;
                    }
                    domRows[m.Price].IncrementVolume(ConvertVolume(m.Volume));
                }
            }
        }

        protected override void OnBarUpdate()
        {
            Print("==================== BIDS ===========================");
            foreach (var kvp in domRows.Where(kvp => kvp.Value.GetMarketDataType() == MarketDataType.Bid))
            {
                var row = kvp.Value;
                Print($"[{row.GetPosition()}]: Price:{row.GetPrice()}, Depth: {row.GetDepth()}, Volume: {row.GetVolume()}");
            }
            Print("======================================================");

            Print("==================== ASKS ===========================");
            foreach (var kvp in domRows.Where(kvp => kvp.Value.GetMarketDataType() == MarketDataType.Ask))
            {
                var row = kvp.Value;
                Print($"[{row.GetPosition()}]: Price:{row.GetPrice()}, Depth: {row.GetDepth()}, Volume: {row.GetVolume()}");
            }
            Print("======================================================");
        }

        private int ConvertVolume(double vol)
        {
            return (int)(Instrument.MasterInstrument.InstrumentType == InstrumentType.CryptoCurrency
                ? Core.Globals.ToCryptocurrencyVolume((long)vol)
                : vol);
        }

        private class DomRow
        {
            private int Depth;
            private int Volume;
            private double Price;
            private int Position;
            private MarketDataType MarketDataType;

            public DomRow(int depth, MarketDataType marketDataType, double price, int position)
            {
                Depth = depth;
                MarketDataType = marketDataType;
                Volume = 0;
                Position = position;
                Price = price;
            }

            public MarketDataType GetMarketDataType() => MarketDataType;

            public void SetData(int depth, int position)
            {
                Depth = depth;
                Position = position;
            }

            public int GetDepth() => Depth;

            public double GetVolume() => Volume;

            public double GetPrice() => Price;

            public void IncrementVolume(int volume) => Volume += volume;

            public int GetPosition() => Position;
        }
    }
}
​