Announcement

**bltdavid** · 08-06-2016, 08:06 PM

Originally posted by ETFVoyageur View Post

Specifically, the property's ToString() method is called before everything is initialized. I contend that NT has no business doing any such thing.

Do you have an NT7 version of this indicator running?
If so, any problems there?

The reason I ask is that ToString() seems to be called relatively early in NT7.

For example, right-click inside a chart and select "Indicators" in the context menu and before the popup dialog appears, the ToString() of every indicator in the list has been called.

I may be wrong, but I'm pretty sure that's what happens in NT7.

Is it not reasonable to assume that ToString() might be called similarly in NT8?

Just my 2�.

**ETFVoyageur** · 08-06-2016, 08:19 PM

No, this involves some code that is new for NT8 and I do not foresee back-porting it.

You are right about timing oddities in NT7. ToString() was one of them, and another was calling into an object before running its constructor -- we had a long discussion about that one; I ended up putting in defensive code.

In the case at hand I found a work-around, so it is not holding me up. It is odd -- the workaround implies that the indicator exists but was never constructed. I still think it s wrong to go calling code before everything is constructed. Just plain basic OOP. The bug should be fixed.

**bltdavid** · 08-06-2016, 08:31 PM

Hmm, I think I may be wrong.

Some quick experimenting in NT7 shows that I might have been confused by the situation where someone sets the "Name" property in Initialize().

Code:

 
 this.Name = "Hello World";   // <-- note spaces are ok

After recompiling, the popup dialog list for Indicators will show an indicator named "Hello World" in the list.

Only after clicking on "New" to add the indicator the chart (and access its properties in the dialog) is ToString() called. That is, the list of indicators added to the chart as seen in the lower left corner is built by calling ToString() for each indicator.

**bltdavid** · 08-06-2016, 08:35 PM

Originally posted by ETFVoyageur View Post

It is odd -- the workaround implies that the indicator exists but was never constructed. I still think it s wrong to go calling code before everything is constructed.

Interesting.

Define "constructed".

Perhaps your definition of "constructed" is not the same as NT's definition.

**ETFVoyageur** · 08-06-2016, 09:14 PM

Originally posted by bltdavid View Post

Interesting.

Define "constructed".

Perhaps your definition of "constructed" is not the same as NT's definition.

My definition is the constructor has run. An object is, by definition, not open for business prior to one of its constructors having run. That's why it is called a "constructor".

**ETFVoyageur** · 08-07-2016, 07:59 AM

Sigh .... it turns out the workaround I said I have is not fully adequate. It does catch the serialization // deserialization cases, but I have discovered there still more cases of NT making calls before everything is initialized. I have now added checks for that reference being null and skipping the functionality if that is the case. I REALLY wish NT would not call the object prematurely!

**koganam** · 08-07-2016, 06:40 PM

Originally posted by ETFVoyageur View Post

My definition is the constructor has run. An object is, by definition, not open for business prior to one of its constructors having run. That's why it is called a "constructor".

Not quite it would seem. When you try to use an indicator, NT makes a copy of an object that it preconstructs. That is why you can reference/assign any of the indicator objects without using the new keyword. That makes things a bit wonky. Of course, in order to populate the PropertyGrid, NT needs to call ToString() early enough to be able to populate the indicator-selection dialog. It appears to do so from the object that it preconstructs.

**bltdavid** · 08-07-2016, 09:45 PM

Originally posted by ETFVoyageur View Post

My definition is the constructor has run. An object is, by definition, not open for business prior to one of its constructors having run. That's why it is called a "constructor".

A constructor is *never* guaranteed to have been called.

NinjaTrader has no examples of any kind of a strategy or indicator using a constructor inside the class definition.

As koganam pointed out, NT might very well re-use a cached instance of your indicator -- in which case, it is the Initialize() and OnStartUp() methods that are used to set things right -- in lieu of the constructor.

If you dare tread deeper in NinjaTrader uncharted waters, take a serious study of the generated code at the bottom of *every* indicator. Then briefly look at bin/Custom/Strategy/@Strategy.cs -- you'll note that the cached list of indicators is being maintained as part of the indicator's generated code.

Due to the cache of indicator objects, NinjaTrader makes no guarantee that it will call your constructor. Best practice is to avoid using them (or at least avoid relying on them.)

If you think about it, NinjaTrader has gone to considerable trouble to create a separate cache of previous instances for each indicator. They must have had a serious desire to limit the construction of so many indicator objects.

Hmm, I wonder why that is ... ?

Answer:
The design decisions behind the generated code probably date back to days of Windows XP when memory was more of an issue.

[Edit: And performance ... they must have noticed a remarkable increase in performance by using that cache idea ... which says something about the slowness of .NET's garbage collector? Or maybe the cache idea fixed GC thrashing ... who knows.]

**NinjaTrader_Brett** · 08-08-2016, 09:46 AM

BLTDavid,

Your pretty much on the money. There are several cases where we will re-use an instance and not construct a new one. This gives us performance benefits along with other cases.

In general we don't recommend user work with constructors at all nor do we document such use with NinjaScript. In lieu of constructors we do support and recommend the OnStateChanged event system with the SetDefaults and Configure events.

If you start to work with constructors you'll start to see lots of little unexpected items that are hard to debug and hard to explain. As I recall you ran into a few months ago since I recall having similar discussion with you on this topic.

I highly recommend move all constructor code to SetDefault if you want it to run prior to the user/system configuring the object and State == Configure if you want code run after the object has been added by the user or deserialized/restored by NT prior to it running.

**bltdavid** · 08-08-2016, 11:11 AM

Originally posted by koganam View Post

When you try to use an indicator, NT makes a copy of an object that it preconstructs. That is why you can reference/assign any of the indicator objects without using the new keyword.

I'm not sure I'd say it like that. A careful study of the generated code at the bottom of the indicator reveals that new is called very sparingly to create a new object of the indicator.

That is, if the cache lookup fails to locate a previous instance with the same exact argument values, only then is a new instance of the indicator created, and after that new object reference is added to the cache, the indicator reference is returned to the caller.

Let's study a specific example.

It is precisely because of this cache lookup & return mechanism that NT7's @SampleMACrossover.cs can assign the colors to the SMA plots and add the same SMA objects to the charts -- with nothing seemingly to connect or guarantee that the two are related. Huh? What magic is this? How are we allowed to expect this to "just work"? I mean, the code never called 'SMA myref = new SMA();', like we normally would in the raw world of OOP. What is really happening here?

It is all done via the method called 'SMA' located in the generated code at the bottom of @SMA.cs -- that generated 'SMA' method is the cache lookup code. It is the magic, the missing piece, it is the middleman that hides the C# call to 'new SMA()'.

This technique is actually quite clever, brilliant even. It may look like magic, but it is clear that cache concept guarantees and explains how the SampleMACrossover's Initialize() code works in NT7, (blue (and red) comments are mine)

Code:

        protected override void Initialize()
        {
            [COLOR=Blue]// this 'SMA' is a method generated automatically during compilation,[/COLOR]
            [COLOR=Blue]// it lives at the bottom of SMA.cs in the NinjaTrader generated code section
            // 'SMA' looks in the cache for previous instance, otherwise calls 'new SMA'
[/COLOR]             SMA(Fast).Plots[0].Pen.Color = Color.Orange;  [COLOR=Blue]// <-- new SMA object #1[/COLOR]
            SMA(Slow).Plots[0].Pen.Color = Color.Green;  [COLOR=Blue]// <-- new SMA object #2
            [COLOR=Red]// Actually, there is no guarantee of 'new' being called here for these objects[/COLOR]
            [COLOR=Red]// If some other strategy or indicator has already loaded an SMA with the[/COLOR]
            [COLOR=Red]// same period value as Fast or Slow, then [U][I][B]that[/B][/I][/U] cached object is returned[/COLOR]
            [COLOR=Red]// rather than creating a new one[/COLOR]

[/COLOR]            Add(SMA(Fast));  [COLOR=Blue]// <-- cache lookup returns SMA object #1[/COLOR]
            Add(SMA(Slow)); [COLOR=Blue]// <-- cache lookup returns SMA object #2[/COLOR]

            CalculateOnBarClose    = true;
        }

The middleman method called 'SMA' is the glue ('SMA' method is supplied by the special code generated during compilation) that allows you to lookup your reference to an SMA indicator object and return the same object every time.

In other words, when programmers write a new indicator, say BestFibonacci, they write a class for the indicator, and they name that class "BestFibonacci".

During compilation, it is the NinjaTrader compiler that generates a method "BestFibonacci" that is used to manage (and create) cached references to objects of the class "BestFibonacci". This generated code is automatically added to the bottom of the indicator's .cs file -- with dire comments warning against removal.

[Side note: In NT7, your Indicator class file must have a Description attribute, or the compiler will not generate this middleman code. Ever wanted to write your own custom cache lookup code? Well, the Description attribute is your knob to turn off the compiler's generation of a "BestFibonacci" method so that you can edit/customize/supply your own "BestFibonacci" method.]

Extremely clever use of .NET features and namespace rules, if you ask me.

Originally posted by koganam View Post

That makes things a bit wonky. Of course, in order to populate the PropertyGrid, NT needs to call ToString() early enough to be able to populate the indicator-selection dialog. It appears to do so from the object that it preconstructs.

My testing in NT7 shows that ToString() is not called to populate the list of available indicators in the indicator selection dialog (class names are used for that list).

[Edit: Well, technically, class names are not used, per se. It is the Name property for an instance of the class (which is usually the same string as the class name itself) that goes into this list. Eg, if the class name is "SMA" but this.Name="YoYoMama" is in Initialize() then "YoYoMama" is what appears in the list, not "SMA".]

Once the "New" button is pressed, the string from ToString() is used to populate lower left panel of selected indicators added to the chart.

That's why an indicator might have one name in the top left selection panel, but then show up as a different name in the bottom left selected panel.

[Edit: Why? ToString() normally just returns this.Name property, but it doesn't have to. If ToString() returns something other than this.Name, then you'll see a difference. Eg, if this.Name="YoYoMama" is in Initialize() but ToString() returns this.Name+"IsUgly" then "YoYoMamaIsUgly" is what appears in the lower left panel, while "YoYoMama" is in the upper left panel.]

**koganam** · 08-08-2016, 11:56 AM

Originally posted by bltdavid View Post

I'm not sure I'd say it like that. A careful study of the generated code at the bottom of the indicator reveals that new is called very sparingly to create a new object of the indicator.

That is, if the cache lookup fails to locate a previous instance with the same exact argument values, only then is a new instance of the indicator created, and after that new object reference is added to the cache, the indicator reference is returned to the caller.

Let's study a specific example.

It is precisely because of this cache lookup & return mechanism that NT7's @SampleMACrossover.cs can assign the colors to the SMA plots and add the same SMA objects to the charts -- with nothing seemingly to connect or guarantee that the two are related. Huh? What magic is this? How are we allowed to expect this to "just work"? I mean, the code never called 'SMA myref = new SMA();', like we normally would in the raw world of OOP. What is really happening here?

It is all done via the method called 'SMA' located in the generated code at the bottom of @SMA.cs -- that generated 'SMA' method is the cache lookup code. It is the magic, the missing piece, it is the middleman that hides the C# call to 'new SMA()'.

This technique is actually quite clever, brilliant even. It may look like magic, but it is clear that cache concept guarantees and explains how the SampleMACrossover's Initialize() code works in NT7, (blue (and red) comments are mine)

Code:

        protected override void Initialize()
        {
            [COLOR=Blue]// this 'SMA' is a method generated automatically during compilation,[/COLOR]
            [COLOR=Blue]// it lives at the bottom of SMA.cs in the NinjaTrader generated code section
            // 'SMA' looks in the cache for previous instance, otherwise calls 'new SMA'
[/COLOR]             SMA(Fast).Plots[0].Pen.Color = Color.Orange;  [COLOR=Blue]// <-- new SMA object #1[/COLOR]
            SMA(Slow).Plots[0].Pen.Color = Color.Green;  [COLOR=Blue]// <-- new SMA object #2
            [COLOR=Red]// Actually, there is no guarantee of 'new' being called here for these objects[/COLOR]
            [COLOR=Red]// If some other strategy or indicator has already loaded an SMA with the[/COLOR]
            [COLOR=Red]// same period value as Fast or Slow, then [U][I][B]that[/B][/I][/U] cached object is returned[/COLOR]
            [COLOR=Red]// rather than creating a new one[/COLOR]

[/COLOR]            Add(SMA(Fast));  [COLOR=Blue]// <-- cache lookup returns SMA object #1[/COLOR]
            Add(SMA(Slow)); [COLOR=Blue]// <-- cache lookup returns SMA object #2[/COLOR]

            CalculateOnBarClose    = true;
        }

The middleman method called 'SMA' is the glue ('SMA' method is supplied by the special code generated during compilation) that allows you to lookup your reference to an SMA indicator object and return the same object every time.

In other words, when programmers write a new indicator, say BestFibonacci, they write a class for the indicator, and they name that class "BestFibonacci".

During compilation, it is the NinjaTrader compiler that generates a method "BestFibonacci" that is used to manage (and create) cached references to objects of the class "BestFibonacci". This generated code is automatically added to the bottom of the indicator's .cs file -- with dire comments warning against removal.

[Side note: In NT7, your Indicator class file must have a Description attribute, or the compiler will not generate this middleman code. Ever wanted to write your own custom cache lookup code? Well, the Description attribute is your knob to turn off the compiler's generation of a "BestFibonacci" method so that you can edit/customize/supply your own "BestFibonacci" method.]

Extremely clever use of .NET features and namespace rules, if you ask me.

My testing in NT7 shows that ToString() is not called to populate the list of available indicators in the indicator selection dialog (class names are used for that list).

Once the "New" button is pressed, the string from ToString() is used to populate lower left panel of selected indicators added to the chart.

That's why an indicator might have one name in the top left selection panel, but then show up as a different name in the bottom left selected panel.

You may not have looked deep enough at the wrapper code in the indicator. If you do you will see that even before cache lookup, the object will still need to be created. How does NT do it? By creating an object using the new keyword, and then making that object part of the base class from which the indicators inherit. It is because the indicator object now becomes part of the IndicatorBase class, that you can get the indicator object without yourself creating a new one using a constructor. When you do so, standard inheritance mechanisms gives you a copy of that premade object. It is a very neat trick, as it in effect turns indicator object creation into a method of the IndicatorBase class, but it does cause some wonky behavior if one is unaware of it.

In my experience, if one insists, one can always write their own constructors for the class, and call those instance or default constructors in the standard OOP manner. I do it often, when I want to break NT's forced ordering of parameters in alphabetical order in the instance constructor. That way, when I need it, I can enforce my own order that makes more sense for what I want to do.

I am well aware of the caching mechanism, which has its own issues, by the way. Object creation is a different kettle of fish: the object must be created or exist before it can be cached.

**bltdavid** · 08-08-2016, 12:38 PM

Originally posted by koganam View Post

You may not have looked deep enough at the wrapper code in the indicator. If you do you will see that even before cache lookup, the object will still need to be created. How does NT do it? By creating an object using the new keyword, and then making that object part of the base class from which the indicators inherit. It is because the indicator object now becomes part of the IndicatorBase class, that you can get the indicator object without yourself creating a new one using a constructor. When you do so, standard inheritance mechanisms gives you a copy of that premade object. It is a very neat trick, as it in effect turns indicator object creation into a method of the IndicatorBase class, but it does cause some wonky behavior if one is unaware of it.

No, if an instance to the indicator is found in the cache, it is the cache lookup code that returns the existing reference. If an instance is not found, a new one is created with 'new', it is initialized and added to the cache, and then returned. Class inheritance has nothing much to do with it, most of the magic comes from the cache feature built into the generated middleman SMA method, which is added to the Indicator (not IndicatorBase) class as a method.

(Edit: More magic is supplied by the Indicator class being defined as "partial".)

(Edit2: I think by "inheritance" you mean "access to". Since SMA is a class which inherits from class Indicator, it gains access to all the public & protected members of the Indicator class -- but the generated SMA public method is still supplying all the cache lookup and object creation magic behind the scenes -- you don't get that for free, someone in Denver had to invent that.]

There is only one actual "premade" object,

Code:

        private static SMA checkSMA = new SMA();

which is used for lock() to protect the cache management and object creation code.

All other objects are either created inside that lock() code as needed (because the cache lookup fails) or a reference to a previous instance (aka, your "premade") is found and returned.

I think we're more close to saying the same thing than not, but I like the term "reference to a previous instance", since it is bit more exact than "premade object".

Also, I'm looking at NT7 generated code in SMA.cs, perhaps NT8 is different?

**bltdavid** · 08-08-2016, 12:46 PM

Originally posted by koganam View Post

If you do you will see that even before cache lookup, the object will still need to be created.

Cache lookup happens first.

If cache lookup fails, only then is 'new' called.

**bltdavid** · 08-08-2016, 01:06 PM

Originally posted by koganam View Post

By creating an object using the new keyword, and then making that object part of the base class from which the indicators inherit. It is because the indicator object now becomes part of the IndicatorBase class, that you can get the indicator object without yourself creating a new one using a constructor. When you do so, standard inheritance mechanisms gives you a copy of that premade object.

No, the reference returned by SMA(n), by definition, will be in the cache maintained by the generated SMA() method. You don't get a copy, you get the actual reference.

There is no copying of premade objects. I'm not sure what you mean by this.

Let me be clear: the returned reference may have been created by 'new' or the reference may have been created earlier and found in the cache -- but all new references are added to the cache before the SMA() method returns, so this sentence is absolutely true: all references returned by SMA() method are in the cache maintained by the SMA() method.

(Edit: For benefit of others, the cache is maintained by the SMA() method, not the SMA class, big difference.)

The references returned by the SMA() method are not part of the IndicatorBase class.

They are maintained in a private list inside the 1 instance (I think) of the Indicator class that is created by NinjaTrader at runtime -- I believe that single master instance goes by the global variable name 'Indicators'.

Topics	Statistics	Last Post
How to connect to external library "Html Agility Pack"? by fx.practic Started by fx.practic, 10-15-2013, 12:53 AM	5 responses 5,403 views 0 likes	Last Post by Bidder Today, 12:22 AM
Trying to add HtmlAgilityPack, run into number of errors by Shai Samuel Started by Shai Samuel, 07-02-2022, 02:46 PM	4 responses 94 views 0 likes	Last Post by Bidder Today, 12:11 AM
Templates/Workspaces created in “Sim” mode - Available in “Live” account? by DJ888 Started by DJ888, Yesterday, 10:57 PM	0 responses 6 views 0 likes	Last Post by DJ888 Yesterday, 10:57 PM
Please Check you Account Connection Parameters and Internet Connection by MacDad Started by MacDad, 02-25-2024, 11:48 PM	7 responses 158 views 0 likes	Last Post by loganjarosz123 Yesterday, 10:23 PM
Trading MES Off of ES Orderflow by Belfortbucks Started by Belfortbucks, Yesterday, 09:29 PM	0 responses 8 views 0 likes	Last Post by Belfortbucks Yesterday, 09:29 PM

Announcement

Partner 728x90

Is there a good explanation of deserialization?

Is there a good explanation of deserialization?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Latest Posts