Using ThreadStatic attribute for multi-threading considerations

[benJephunneh]

Coding for multi-threading is somewhat of a new world for me, and I'm struggling to grasp just how much a NinjaScript coder needs to be mindful of it and in what scenarios.
(Note: All questions are unrelated to coding for graphics or UI objects. I'm considering only calculations and ISeries objects.)

An obvious scenario for taking care would be file I/O, but what about having indicator1's accessing indicator2? Certainly that's too broad a question, so here is an example.


Let's say I start with a basic indicator such as BuySellVolume. Now, I've seen at least one similar example where a lock was placed on the MarketDepthEventArgs object (something like "lock(e.Instrument.MarketDepth.Asks)", etc.). Why place a lock within OnMarketDepth and not OnMarketData? The Multi-Threading documentation mentions that market data is not thread safe such that use of the Dispatcher is recommended, but then none of the built-in indicators accessing market data use the Dispatcher. What am I missing, here?

Anyway, if I create another indicator that creates a BuySellVolume object or some other indicator object and pulls values from it, would the following ever be of any use as a field declaration?

[ThreadStatic] private static BuySellVolume _bsv;

When I access the Buys or Sells series, I shouldn't need to lock since I'm doing only reads (right?), but then the Buys and Sells series are being written to without a lock... Is it that BuySellVolume could be improved for better thread safety, or is it that these kinds of multi-threading considerations are taken care of under the hood (I seem to remember something about ISeries being ConcurrentDictionaries?)?

Perhaps relatedly, why do some NT 8 indicators call Update() in the getters of Values series and others do not? Is there a reason to not always add an Update call to every getter?

Can it be explained to what extent multi-threading should be considered, or rather, what are the weaknesses that I need to consider? I mention Values series (e.g. those created from AddPlot( ...args... )) because those series would more obviously create conflicts considering the potentially synchronous reads and writes. What about a custom Series object? I see those are created using the ISeries interface, so are locks for them similarly unnecessary?

Is multi-threading an important consideration for only non-State methods? Do you use locks? Monitors? Tasks?

Are there any examples where one's simply calling another indicator's Values series will cause a multi-threading conflict even if the other indicator runs calculations in only OnBarUpdate? With OnEachTick? OnBarClose? IsFirstTickOfBar?





I apologize for cramming so many questions in there. I understand this is a very broad topic, but I feel the NT8 documentation is either too light on the subject or doesn't clearly explain the scope of the problem (e.g. it's for mainly rendering considerations and most everybody else can stop reading). If anybody has examples or even some good reading material, I'd greatly appreciate it. I'm fairly certain I've had threading bugs on even basic indicators (no fancy drawing, no rendering, just Values[0] = 1 + 1 kind of stuff) being called from other indicators before I learned about multi-threading, but then there doesn't seem to be any relevant NT8 documentation for basic NinjaScript coding. The more I learn about this, though, the more I think lock is not the right tool for the jobs I'm doing.

Many thanks in advance.

Best,
Caleb

[NinjaTrader_Kate]

Hello benJephunneh,

Thank you for your post.

Here's the basics In terms of threads and what you need to know when creating NinjaScript Addons. There are really 3 thread entry points that need to be understood:

UI Thread

NinjaTrader spawns multiple UI threads, it spawns them per the number of logical processor cores detected on startup.

1. Each time a window is opened it will randomly be assigned a UI thread on each startup, in windows a 'window' must be controlled and owned by a single UI thread.
2. Any actions derived from that window are triggered by the UI thread that owns the window.

Instrument Thread

NinjaTrader spawns multiple instrument threads as well, it spawns them per the number of logical processor cores detected on startup.

1. If you have 4 cores this means you'll have 4 UI threads and 4 instrument threads for a total of 8 threads.
2. Once a market data subscription is started, it is assigned an instrument thread randomly and all processes which are driven from real time data for that instrument will be on the same instrument thread moving forward. The instrument thread will then drive logic.

Thread Pool / Adapter Threads

Thread pool is some action you scheduled to execute something temporary in the background using a random thread which is not the instrument thread or the UI thread. We use these is some places in NinjaTrader, most notably during historical processing.

Adapter threads are any thread spawned by the adapter. Each adapter is a little different, an adapter can have a separate adapter thread or multiple separate adapter threads depending on the architecture for whatever best matches the architecture of the third party we integrate with. This would handle things like order submissions.

As far as methods and what threads they are called by:

OnBarUpdate

1. Called by ThreadPool in historical processing.
2. Called by InstrumentThread on each update in realtime processing.
3. Each bars object maintains their own market data subscription, which serves as the subscription to both update the bars + run NinjaScripts. This subscription is different then the subscription done by just a L1 data subscription for displaying some data on a UI. E.g. Chart can have two subscriptions, the bars subscribed and the Chart Trader Ask/Bid volume is subscribed, however both would be driven from the same instrument thread.

OnRender

1. Called by UI Thread

OnMarketData

1. Called by Instrument Thread

OnStateChanged

1. Generally called by UI Thread when adding, modifying, deleting NinjaScript settings from the UI but is considered to be able to be called by any thread. Example:
   1. Called by ThreadPool for State.Historical
   2. Called by Instrument thread for State.Realtime.
   3. Called by UI thread for State.Configure

OnOrderUpdate, OnExecutionUpdate, OnAccountUpdate, OnPositionUpdate, OnConnectionStatus

1. Consider called from non-UI / non-instrument thread. Typically an Adapter Thread.

OnCalculateMinMax

1. Called by UI Thread

Now that we've gotten the basic threading model out of the way, let's tackle your questions.

Let's say I start with a basic indicator such as BuySellVolume. Now, I've seen at least one similar example where a lock was placed on the MarketDepthEventArgs object (something like "lock(e.Instrument.MarketDepth.Asks)", etc.). Why place a lock within OnMarketDepth and not OnMarketData? The Multi-Threading documentation mentions that market data is not thread safe such that use of the Dispatcher is recommended, but then none of the built-in indicators accessing market data use the Dispatcher. What am I missing, here?

OnMarketData runs on synchronously to OnBarUpdate - that's how Tick Replay works - but OnMarketDepth is aysnchronous. So you can think of OnRender, (OnBarUpdate/OnMarketData) and OnMarketDepth as separate threads.

Anything that is shared (read/written) between threads would need a lock. So, OnMarketDepth needs a lock. You would use a Dispatcher to access or modify something on a UI thread.

Anyway, if I create another indicator that creates a BuySellVolume object or some other indicator object and pulls values from it, would the following ever be of any use as a field declaration?
Code:
[ThreadStatic] private static BuySellVolume _bsv;

No, as BuySellVolume is an indicator object, it would not be useful here and would likely cause issues.

When I access the Buys or Sells series, I shouldn't need to lock since I'm doing only reads (right?), but then the Buys and Sells series are being written to without a lock... Is it that BuySellVolume could be improved for better thread safety, or is it that these kinds of multi-threading considerations are taken care of under the hood (I seem to remember something about ISeries being ConcurrentDictionaries?)?

You would not need to use a lock accessing buys or sells in both OnMarketData and OnBarUpdate since they are synchronous and on the same thread.

Perhaps relatedly, why do some NT 8 indicators call Update() in the getters of Values series and others do not? Is there a reason to not always add an Update call to every getter?

In most cases, Update isn't necessary. Really, this would only be needed when you need a specific value updated to the current bar, for example, there's an exposed variable that needs OnBarUpdate to be run to access. You'd really rarely need to use this.

Update (See notes for when more complex version is needed) - https://ninjatrader.com/support/help...nt8/update.htm

OnBarUpdate (See Notes for when Update would be needed) https://ninjatrader.com/support/help...nbarupdate.htm

Can it be explained to what extent multi-threading should be considered, or rather, what are the weaknesses that I need to consider? I mention Values series (e.g. those created from AddPlot( ...args... )) because those series would more obviously create conflicts considering the potentially synchronous reads and writes. What about a custom Series object? I see those are created using the ISeries interface, so are locks for them similarly unnecessary?

Series objects will be synchronized to a bars series and would not need to be locked when accessed within OnBarUpdate or OnMarketData as these would be running on the same thread. You may find situations where you wish to access historical price series outside of the core data event methods, such as OnRender(), or your own custom event. In these advanced scenarios, you may run into situations where the "barsAgo" pointer is not in sync with the current bar, and may result in errors when trying to obtain this information. In those cases, please use the Bars.Get...() methods with the absolute bar index, e.g., GetValueAt().

Is multi-threading an important consideration for only non-State methods? Do you use locks? Monitors? Tasks?

I'd refer you to how OnStateChange fits into the above threading model above. Can you clarify what you mean by is multi-threading an important consideration for only non-State methods? Monitors and tasks would be considered advanced locking techniques and would be outside the scope of our support.

Are there any examples where one's simply calling another indicator's Values series will cause a multi-threading conflict even if the other indicator runs calculations in only OnBarUpdate? With OnEachTick? OnBarClose? IsFirstTickOfBar?

If you're trying to access the values series in OnRender, you'd want to use GetValueAt() to avoid issues where the bars ago isn't in sync. If you're just calling the indicator within OnBarUpdate you should not run into threading issues accessing a values series from that indicator.

Please let us know if we may be of further assistance to you.

[benJephunneh]

Kate, thanks so much for your walkthrough and all your considerations. I really appreciate the effort. I can't digest all of that in one read, so I'll chew on this for a while and let you know as more questions materialize.

Again, much appreciated!