Strategy Threading...?

[NetTecture]

Hello,

a simple question. Is a strategy Single Threaded in it's behaviors or not?

I.e., does NInjaTrader synchronize access to the strategy methods (so that only one method is ever actively called), or do I have to sync that myself, if my method is not reentrant?

I mostly ask becaue the documentation refers to multi threaded reportings for fills. Now, in the OnOrderUpdate override - can I assume that this is single threaded when called (i.e. one update after the next), or do I have to code that thread safe (possibly by just wrapping it into a lock). I start doing quite some complicated stuff there, and quite a lot of the .NET base classes (whole collection namespace) are not thread safe per definition unless explicitely asked for.... Thus my question.

[NinjaTrader_Dierk]

It's single threaded for NT6.5 but not for NT7. Your code should not rely on any particular threading.

[NetTecture]

It's single threaded for NT6.5 but not for NT7. Your code should not rely on any particular threading.

Ah, ok thanks. Good enough answer for me for now.

That said - personally I think that decision for NT7 is a pretty badly thougt out one.

Is there any particular reason to dump the complexity up to the strategy programmer to code reentrant strategies?

For me, this basically will mean (NT7) to put a lock(this) in every method. Strategy execution is not exactly time sensitive (as in: it should happen as fast as possible, but the individual methods do not really take time), so that the overhead of getting in and actually programming thread safe methods is simply and per definition not worth it. I can not have strategy decision code (OnBarUpdate) blow because it goes through an order enumaration... which was changed because right at this moment I got an order execution.

It can be a hassle, and is simply not worth the time (programmer time vs. performance benefit). Note that I do not say "do not use multi threading in NinjaTrader", what I say is "make sure that in a strategy instance, there only and ever is one event/method active at a time", similar to the ASP.NET model that has up to (standard) 25 threads per processor... but one page object is only and always handled by one single thread at a time.

Otherwise: you are in for a world of pain. Seriously. A lot of the .NET framework is not thread safe - the whole collections namespace comes to my mind.

[NinjaTrader_Dierk]

Thanks for your suggestion. We'll add it to the list of future considerations.

[NetTecture]

Note there is a big reason for that.

Let's assume I have a list of IOrder. Not exactly something uncommon. Getting to an IOrder is a little tricky, so I have a Dictionary<string,IOrder> that maps an IOrder TOKEN to the IOrder.

In OnBarUpdate I am cancelling some orders.

foreach (string token in _MyOrders.Keys) {
IOrder order = _MyOrders[token]
....
CancelOrder (order);
}

As simple as it gets

NOT... if I have an handler in OnOrderUpdate that automatically removes order that are finished (canclled, filled).

Because if you code that non-reentrant.... then the OnOrderUpdate method has changed the collection possibly DURING the loop, resulting in a nice exception that the collection was changed (which MS incidentally tracks - collections have version numbers, the numerator generated checks that one. One reason you can not use the above loop to REMOVE order from the collection).

This is terribly itchy to program around

[NinjaTrader_Dierk]

Your issue is not related to threading rather than if OnOrderUpdate would be called while CancelOrder is executed (even in the same thread). The answer to that question is: your code needs to be prepared for that and make sure it would not corrupt your self managed dictionary.

[NetTecture]

Actually it is, because if you have single threaded implementation, then there simply is no way to get an update WHILE IN ANOTHER METHOD.

Basically, in this example, the OnOrderUpdate method call would wait until the OnBarUpdate execution has finished.

You dont have to even look far to see an implementation of this: WinForms has the same - one thread. Can e "one thread per window" actually (if one reads up how to open more message pumps). It basically guarantees that executions within a window are single threaded.

THe only sane and fast way for the developer to implement this is a "lock(this) in basically every method handler. Which is a lot more expensive than you doing that outside.

[NinjaTrader_Dierk]

>> Actually it is, because if you have single threaded implementation, then there simply is no way to get an update WHILE IN ANOTHER METHOD.
Incorrect.

foreach (object o in yourContainer)
     Method1();       // now Method1 internally calls Method2, no threading involved

private void Method2()
{
     yourContainer.Add(someObject);    // problem, since it would corrupt "yourContainer"
}

[sefstrat]

You can run into threading issues even in NT 6.5 (for instance if you use IOrders with the System.Collections namespace, which as you mentioned is not thread safe)

This kind of problem will not show up in backtesting or market replay because they are always run on a single thread, whereas the live execution engine is multithreaded.

[NinjaTrader_Dierk]

Let's not get confused:
a) NT6.5 strategy methods are all executed in the same thread, no matter if live, or back test or replay
b) this will change will NT7
c) issue on collections can arise (and will arise) in the scenario as per my prior post even without any threading involved. No "lock" will help to resolve whatsoever.

-> whenever you manage e.g. IOrders in your own collection you need to make sure that the collection will not be changed as you return the control back to NT by calling a NinjaScript method, since NT might call NinjaScript callbacks where your code then might manipulate (and corrupt) that collection.

[sefstrat]

So you claim that the live execution engine is not multithreaded?

Is it not what invokes strategy.OnExecution?

I have run into several situations that exhibit different behvaior live vs backtesting or replay due to issues related to threading and the internal order handling rules.

[NetTecture]

I leave it like that. Pretty obvious that whoever did the planning at the side of NinjaTrader did not bother getting into common approaches and problems and dumps it all on the strategy developer.

Bad design. Not the first time I see that. We will have to live with it. At least NT 6.5 is better in this regard.

Btw., Dierk, you may realize that "calling a method changing a collection" is inherently something different than "getting an update asynchroneusly". Funny how people in NinjaTrader support are unaware of the support nightmares multi threaded callbacks produce.... especially in scenarios where those are totally superflous (because, seriously, I can never see a strategy requiring multi threading in itself and it's direct handling).

[NinjaTrader_Dierk]

So you claim that the live execution engine is not multithreaded?

Is it not what invokes strategy.OnExecution?

I have run into several situations that exhibit different behvaior live vs backtesting or replay due to issues related to threading and the internal order handling rules.

6.5: NinjaScript methods are all called in the same thread, regardless if the actual broker adapter code is multi-threaded or not.

[NinjaTrader_Dierk]

I leave it like that. Pretty obvious that whoever did the planning at the side of NinjaTrader did not bother getting into common approaches and problems and dumps it all on the strategy developer.

Bad design. Not the first time I see that. We will have to live with it. At least NT 6.5 is better in this regard.

Btw., Dierk, you may realize that "calling a method changing a collection" is inherently something different than "getting an update asynchroneusly". Funny how people in NinjaTrader support are unaware of the support nightmares multi threaded callbacks produce.... especially in scenarios where those are totally superflous (because, seriously, I can never see a strategy requiring multi threading in itself and it's direct handling).

Please re-read my posts below to understand that your code in post #5 might be flawed regardless of any threading involved.