Multithread Data Access

[Curerious]

Hi,

If I am modifying some object data within the same script but under different methods, should I introduce lock object for multithreading safety or they are fine? I understand that NT8 uses multithreading internally for different methods, but I didn't see much of the built-in indicators uses any lock objects.

    public class TestIndicator2 : Indicator
    {
        private Data _data;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Enter the description for your new custom Indicator here.";
                Name                                        = "TestIndicator2";
                Calculate                                    = Calculate.OnEachTick;
                IsOverlay                                    = false;
                DisplayInDataBox                            = true;
                DrawOnPricePanel                            = true;
                DrawHorizontalGridLines                        = true;
                DrawVerticalGridLines                        = true;
                PaintPriceMarkers                            = true;
                ScaleJustification                            = NinjaTrader.Gui.Chart.ScaleJustification.Right;
                //Disable this property if your indicator requires custom values that cumulate with each new market data event.
                //See Help Guide for additional information.
                IsSuspendedWhileInactive                    = true;
            }
            else if (State == State.DataLoaded)
            {
                _data = new Data();
            }
        }

        protected override void OnBarUpdate()
        {
            _data.Set_Data(1,0.1);
        }

        protected override void OnMarketDepth(MarketDepthEventArgs marketDepthUpdate)
        {
            base.OnMarketDepth(marketDepthUpdate);
            _data.Set_Data(1, 0.1);
        }

        private class Data
        {
            private SortedDictionary<double, double> _Data;
            public void Set_Data(double key, double value)
            {
                if (this._Data.ContainsKey(key))
                    this._Data[key] += value;
                else
                    this._Data[key] = value;
            }

            public Data()
            {
                this._Data = new SortedDictionary<double, double>();
            }
        }
    }​

[NinjaTrader_Jesse]

Hello Curerious,

That generally won't be required for items you create in your script however can still use locks if you need to make sure that both methods are not writing to the same datapoint in your collection.

[Curerious]

Thanks Jesse

[Curerious]

NinjaTrader_Jesse

Hey Jesse,

I have two following questions:

1. Are the override methods in NT8 thread safe? ( For example, Will OnBarUpdate and OnStateChange be called at the same time? Will OnBarUpdate and OnRender be called at the same time? Will other override methods be called at the same time? )

2. If the override methods in NT8 is not thread safe, what if I access Bars object or Values object in different override methods ? Are those objects build-in thread safe or I have to use lock(Bars) / lock(Values)? (For example, Will the data crush if I access Bars.GetClose or ISeries<T>.GetValueAt or Values.GetValueAt across different override methods ? Since if they got changed at OnBarUpdate I assume that may creates read/write competition at override methods such as OnRender / OnOrderUpdate.)

Thank you !

[NinjaTrader_Jesse]

Hello Curerious,

The overrides and properties in your script are thread safe assuming you are not working with external items like the GUI. OnStateChange is called for specific state changes where OnBarUpdate is called during processing of data.

[Curerious]

Thank you for replying Jesse, Here comes three following questions:

1. What specifically do you mean by "external items" ? Can you provide me an exhaustive list ? Because I see the built-in indicators used locks.. such as Volumes.cs.

2. Are you hinting that if I do not with external items, I do not need to introduce locks anywhere within my script?

3. How is working with external items changes the design of my script? Are you hinting that if I work with the external items then the overrides and properties become thread unsafe? How is one need to handle those situations then?

[NinjaTrader_Jesse]

Hello Curerious

There is no exhaustive list but one sample would be working with the chart GUI like the chart trader from your script.

The superdom volume.cs column works differently that an indicator, as we were speaking about an indicator that was not relevant.

Locks are only necessary in very specific use cases, you very likely will never need to use a lock in an indicator if you are using the override methods for processing and local variables which reside in your script.

[Curerious]

Thank you Jesse !

The reason I asked what I asked is because I saw many other NT8 forum claimed that the use of locks is important. But the little that I know is : those were specified under different categories of development, such as AddOns or SuperDomColumns, which the use of locks usages were introduced under those specific documentations.

[NinjaTrader_Jesse]

Hello Curerious,

Yes its really when you are accessing something external to the script. An example would be the APQ superdom column. In that script it accesses lock(SuperDom.MarketDepth.Instrument.SyncMarketDep th) which is a collection external to the column, that locks the collection so an operation can be performed on the collection. The reason a lock is used there is that script is using a linq expression which translates down to a foreach loop over the collection, if the collection is modified while its looping it will cause an exception. An example in your script would be using a C# List and then adding and removing from the list on each tick while also trying to access the list and loop over it in another location like a seperate indicator. You would need a lock before looping over the list to avoid collection modified exceptions.

[Curerious]

Thanks for the information Jesse !

So is that ok to say that NT8 is not concurrent within a single script? The script is serial execution based (One Thread At a Time) ?

An example in your script would be using a C# List and then adding and removing from the list on each tick while also trying to access the list and loop over it in another location like a seperate indicator.

Also, regarding to this statement, I often time see people accessing Values properties or other ISeries<T> from an indicator, but those indicator did not uses locks or what so ever to protect read/write conflicts, how does that work? What if I exposed a public method to access the local field, I need to used lock to access that local field correct?

[NinjaTrader_Jesse]

Hello Curerious,

NinjaTrader handles all the threading its really only your own code that can come into question, like using a C# List and foreach loops in C# which are prone to exception which another thread is modifying the collection.

A List is not a Series, series are thread safe. You can't do C# list operations on a series or use foreach loops on a series. Values is just a collection of Series<double> objects so also the same category which doesn't require lock.

You can use external tools like vs code to search all files in the custom folder for lock( and that will show you the very few places it is used, it is extremely infrequent that you need to use that.

[Curerious]

You are right Jesse, thank you for your patience. I will just take your word for it.