lightweight as possible.

[ETFVoyageur]

I have re-read the lifecycle and best practices pages and still have questions. My basic question is just how and when indicator data gets initialized. Data gets initialized in several ways: at compile time, by the indicator's code, by un-serializing saved data, and by user interaction with the configuration dialog. I do not fully understand how those ways interact and how they interact with the early state changes.

Compile time -- the documentation does not discuss this, but I would think that is the lightest weight way for data to get initialized.

Constructor -- I understand the advice to keep this as light as possible. As far as I know, that is because it helps the system for constructors to be as light as possible. After all, the constructors are run a lot. Also, for architectural reasons, some things cannot be done there. For example: serialized data and data that depends on system items that are not yet available. I don't see any architectural reason why other data cannot be initialized there -- I see only the lightweight reason, which is good enough for me, so I'll be a good citizen and postpone data initialization.

SetDefaults -- This is my major question -- what is the state of the data when the code enters SetDefaults? Is there a difference between serializable and non-serializable (XmlIgnore) configurable data? I understand that configurable data must be initialized here and that, to keep it lightweight, other data should not be initialized here. I need a better understanding of the state of serializable data when the code enters this state. Has any de-serialization already taken place, has cloning already taken place, or is the data as I would expect from the compiler plus whatever the code did in the constructor? Can I depend on compile-time initialization, or can SetDefaults be called on a non-virgin instance that really does need runtime initialization?

How far off is my mental model?

• The constructor sees virgin data but should, as discussed above, be kept lightweight. Most initialization should be done elsewhere.
• On entry to SetDefaults, the data is still virgin, which suggests to me that compile-time initialization should be fine and the code cannot expect any previous state to be set yet.
• Once the code has done whatever initialization it wants to in SetDefaults, the system will overlay de-serialization.
• Then the user gets to configure whatever is represented in the configuration dialog.
• The next thing the code sees is the Configure state when everything has been melded and the code sees fully-configured data.

I guess what I am looking for is a data lifecycle description analogous to the code state lifecycle description already in the documentation. The single question I care most about today is what state the data is in when the code enters SetDefaults.

Thanks for any guidance.

[ETFVoyageur]

I'm adding to this thread because I have discovered another reason to wonder what state the data is in and at what point.

Tonight’s shock has been discovering that OnStateChange() is called with state SetDefaults before that instance’s constructor has run! I discovered that while chasing down a reference to an uninitialized property, and that turned out to be the cause. Debug code in SetDefaults was referencing a property that the constructor sets up. (The code is robust enough that there is no real problem, but it did make me chase down the anomaly.) It was (is?) hard for me to believe that SetDefaults is running before my constructor, but I have three different ways that all point to the same thing.

The first thing is my debug print statements. I routinely generate a unique ID for each instance of each of my indicators. I have three indicators on my chart while I am doing some debugging. For each debug output I can tell what method of what instance of what indicator it came from. The multiple instance capability is handy in the NT environment where the system routinely creates multiple instances of an indicator. That output shows without a doubt that SetDefaults is running before that instance’s constructor.

The second thing, consistent with the above although I suppose not really proving anything is that NinjaScript.State is “SetDefaults” when the constructor is called. I had noticed that some time ago and just assumed it was a system peculiarity. Now I wonder – my guess is that it is set that way when SetDefaults is called and remains that way until it switches to Configure. Perhaps no proof, but quite the coincidence if not.

The third way is definite proof. I set two instance bools: ConstructorAlreadyRan and SetDefaultsAlreadyRan. Each of them is initialized to false. SetDefaults and the constructor each turn their own bool true and check the other one. SetDefaults finds the constructor one still false. The constructor finds the SetDefaults one has indeed already been set true. I’d call that pretty clear proof.

So how can that be? It’s certainly counter to what anyone who is used to C# or C++ or etc. expects! Perhaps the constructor in use is one from the magic code wrapper that then calls on SetDefaults, followed by calling the real constructor? However it is happening, it seems to me like a very strange architecture, to put it kindly. Aside from needing to fix my code to allow for this out-of-order execution, I wonder what other surprises await. This one is still hard to believe, but with the above three points I have to believe unless someone has a good explanation I have not thought of.

The other big thing, and the reason for posting in this thread, is that it reinforces my request to know what the state of the data is at any point, at least up through the Configure state (I would hope things have settled down by that point).

--EV

[bltdavid]

NinjaScript is not magic, it is simply C# code.

The method OnStateChange (just like every C#
method) cannot execute before the constructor.

There are multiple instances of your indicator being
created/cloned before it ever appears on your chart.

If you could attach your test script, I'm sure all of the
behavior you see could be explained.

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

There is a video you may find helpful.
Understanding NinjaScript Lifecycle and How to Use OnStateChange() in NinjaTrader - https://www.youtube.com/watch?v=gyel6m3VIWs

And a forum post that discusses the clones created for populating the UI.

https://ninjatrader.com/support/forum/forum/historical-beta-archive/version-8-beta/85086-why-is-state-terminated-run-when-another-different-indicator-is-loaded?p=726288#post726288

By indicator data I think you are referring to the property values and not historical or real-time market data, correct?

Compiling can generate clones for the Strategy Analyzer Strategy drop-down list, or for any open Indicator or Strategy windows, to list the new property defaults from State.SetDefaults. This doesn't however reload any running instances of scripts so these will not run through the State.SetDefaults.

Property values saved in xml are pulled when a workspace is opened, or when a template is applied to a new instance. bltdavid is correct, the constructor runs when the instance is generated and always runs first. For any new instances of scripts that have a default template saved, this will be applied after State.SetDefaults and before State.Configure behind the scenes to set the values pull from template and display in the Indicators or Strategies window.
For any existing instances that are being restored from a workspace, these values will be set before State.Configure.

Serializable means that the object type can be converted to a string for saving in an xml text file and restored from the string back to the original object type. If the object is not serializable, it must use the XmlIgnore() attribute and will not be saved in xml.

"Tonight’s shock has been discovering that OnStateChange() is called with state SetDefaults before that instance’s constructor has run!"

This is not correct. You are likely looking at a cloned instance created to populate the UI running SetDefaults and comparing this with the constructor being created for a new instance. It's not possible in C# for any methods of class to run before the constructor. The constructor constructs the class.

[ETFVoyageur]

OK – here is your chance to explain what I am misunderstanding. I have tried to explain my issue as clearly as I can. Please ask if anything is not clear.

=====

To be clear, I understand the above reactions. I, too, believe NT is very high-quality code. I, too, understand that C# always runs some constructor when creating an object. My concern is whether my constructor is being run before the rest of my code.

I woke up this morning realizing that my actual environment is a bit complicated, so I needed to create a simple test case showing the problem. I need to prove (or disprove) that it is nothing about my environment and that I am not misunderstanding anything. I also need a simple test case to show so that others can help me understand what is going on.

I created a new workspace. I then added one new indicator, created by just clicking through the wizard without adding anything. That left me with as simple and fully supported/approved a situation as I know how to create. That, of course, compiled and ran.

Next, I added as little code as I could to investigate the issue. This keeps the test as simple as I know how to make it. The code I added was:

1. Two Booleans, one for the constructor and one for OnStateChange, initialized as false.
2. A default constructor with two lines of code: set its bool and print the other bool.
3. Two lines of code in OnStateChange: set its bool and print the other bool.
4. A comment line above each change so my code changes will be clear in the source.

Here is the output, in this order, when the Indicators dialog is raised:

***** OnStateChange: ConstructorAlreadyRan=False, State=SetDefaults
***** Constructor: OnStateChangeAlreadyRan=True, State=SetDefaults

It is clear from the order of the printed lines, from the printed bool values, and (probably) by the value of “State” in the constructor, that my constructor ran, but after OnStateChange () had already run with State=SetDefaults.

This is why I am worried that OnStateChange may be running before my default constructor. Some constructor must have been run, since it is C#, but it appears it was not my constructor. “ConstructorAlreadyRan” should never be false when entering OnStateChange ().

Another interesting point is that some other constructor must have been run, so I presume mine was subsequently explicitly called as a function call. I have no problem with that (assuming the constructor that actually ran was transparent to my code), but I have a big problem with my constructor being called after OnStateChange () has already been called. I’m not sure about the transparency, though – did that other constructor do such things as initialize configurable parameters? I can even live with non-transparency, as long as I know just what to expect.

And, finally, there is the matter of State being SetDefaults when my constructor is run.

Since the only thing I did was to bring up the Indicators dialog, there should be just one instance of the indicator – the one created to populate the list of available indicators. In any case, this test does not depend on having only a single instance.

Here is the full source code, including the magic wrapper code. I would appreciate any help understanding:

• Why ConstructorAlreadyRan is false when entering OnStateChange ()
• Why “State” should be “SetDefaults” when my constructor is finally run.
• Whether the constructor that actually ran did anything that is visible to my indicator.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Indicators in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Indicators
{
    public class ZZ_ConstructorTest : Indicator
    {
        
        // I added these two boolean variables
        private bool    ConstructorAlreadyRan = false,
                        OnStateChangeAlreadyRan = false;
        
        // I added this default constructor
        public  ZZ_ConstructorTest()
        {
            ConstructorAlreadyRan = true;
            Print("***** Constructor: OnStateChangeAlreadyRan=" + OnStateChangeAlreadyRan.ToString() + ", State=" + State.ToString());
        }
        
        protected override void OnStateChange()
        {
            // I added these two lines of code
            OnStateChangeAlreadyRan = true;
            Print("***** OnStateChange: ConstructorAlreadyRan=" + ConstructorAlreadyRan.ToString() + ", State=" + State.ToString());
            
            if (State == State.SetDefaults)
            {
                Description                                    = @"Trying to reproduce a bug..";
                Name                                        = "ZZ_ConstructorTest";
                Calculate                                    = Calculate.OnBarClose;
                IsOverlay                                    = false;
                DisplayInDataBox                            = true;
                DrawOnPricePanel                            = true;
                DrawHorizontalGridLines                        = true;
                DrawVerticalGridLines                        = true;
                PaintPriceMarkers                            = true;
                ScaleJustification                            = NinjaTrader.Gui.Chart.ScaleJustification.Right;
                //Disable this property if your indicator requires custom values that cumulate with each new market data event.
                //See Help Guide for additional information.
                IsSuspendedWhileInactive                    = true;
            }
            else if (State == State.Configure)
            {
            }
        }

        protected override void OnBarUpdate()
        {
            //Add your custom indicator logic here.
        }
    }
}

#region NinjaScript generated code. Neither change nor remove.

namespace NinjaTrader.NinjaScript.Indicators
{
    public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase
    {
        private ZZ_ConstructorTest[] cacheZZ_ConstructorTest;
        public ZZ_ConstructorTest ZZ_ConstructorTest()
        {
            return ZZ_ConstructorTest(Input);
        }

        public ZZ_ConstructorTest ZZ_ConstructorTest(ISeries<double> input)
        {
            if (cacheZZ_ConstructorTest != null)
                for (int idx = 0; idx < cacheZZ_ConstructorTest.Length; idx++)
                    if (cacheZZ_ConstructorTest[idx] != null &&  cacheZZ_ConstructorTest[idx].EqualsInput(input))
                        return cacheZZ_ConstructorTest[idx];
            return CacheIndicator<ZZ_ConstructorTest>(new ZZ_ConstructorTest(), input, ref cacheZZ_ConstructorTest);
        }
    }
}

namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns
{
    public partial class MarketAnalyzerColumn : MarketAnalyzerColumnBase
    {
        public Indicators.ZZ_ConstructorTest ZZ_ConstructorTest()
        {
            return indicator.ZZ_ConstructorTest(Input);
        }

        public Indicators.ZZ_ConstructorTest ZZ_ConstructorTest(ISeries<double> input )
        {
            return indicator.ZZ_ConstructorTest(input);
        }
    }
}

namespace NinjaTrader.NinjaScript.Strategies
{
    public partial class Strategy : NinjaTrader.Gui.NinjaScript.StrategyRenderBase
    {
        public Indicators.ZZ_ConstructorTest ZZ_ConstructorTest()
        {
            return indicator.ZZ_ConstructorTest(Input);
        }

        public Indicators.ZZ_ConstructorTest ZZ_ConstructorTest(ISeries<double> input )
        {
            return indicator.ZZ_ConstructorTest(input);
        }
    }
}

#endregion

[ETFVoyageur]

Here is a summary of my thoughts. I welcome an explanation of anything I may misunderstand.

• Multiple instances of the indicator are not involved in this discussion. The issue can be shown by simply bringing up the Indicators dialog. As far as I know, there is only a single instance in that case – the one created to populate the list of available indicators. Also, if more than one instance were being created there would be more than one set of printed output lines.
• Of course, some constructor is used to construct each indicator instance. I say again “some” constructor. Nothing about C# guarantees that it is my constructor. It appears that some constructor other than mine is being used to construct the instance. That would imply the system is somehow adding a constructor of its own to my class – I don’t know how but can think of more than one way it could be done.
• It appears that something is in fact calling my constructor, but only after first calling OnStateChange () (with State=SetDefaults). If you doubt that, how do you explain the order of the output lines and the bool values they report?

If that is correct, then I have some further points:

• For this scheme to work, either the constructor that actually gets run needs to be transparent to the indicator or else the indicator needs to know what to expect.
• The most obvious non-transparency is that the call to my constructor is not the first call into my code. Anyone writing code expects their constructor to be the first function that gets run, with any other code being entitled to depend on the constructor being run first. I can work around that if I know what to expect.
• Another non-transparency would be if the constructor being run sets any values that are visible to the indicator. If that is happening, it is important for the indicator to know what to expect.

Thanks to bltdavid for his offer, and I’m looking forward to it -- the test script is in my previous post:

If you could attach your test script, I'm sure all of the
behavior you see could be explained.

[ETFVoyageur]

This discussion brings up a related question: what should NinjaScript.State have for a value before OnStateChange() has been called for the first time? Looking at the choices offered by intellisense in the NinjaScript editor I did not see any reasonable choice.

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

I understand your meaning. The Indicator class constructor could be running first.

I'll ask our development about this and if I get any further information I will pass this on to you.

[ETFVoyageur]

Thank you, Chelsea.

I just need to understand what is happening so that I can do whatever I need to do. I am confident I can make it work as long as I understand what is going on.

I have already fixed my code to work with the calling order. I have assumed that the call to OnStateChange() is the first call made to my code. I need to know whether I am wrong and something else is called first.

I also need to know what state to expect for the class public properties.

[ETFVoyageur]

Another example of a question:

When my constructor finally gets called does its chaining the base constructor still happen? I hope not because that sounds like a double construction of the Indicator class. Still, that's what the constructor syntax says will happen.

[ETFVoyageur]

I hope there is some simple well-defined answer to all of this. However, I am beginning to think that neither normal inheritance nor a constructor should be employed by indicators.

• NT recommends that indicators inherit directly from Indicator (i.e. not use an intermediate base class)
  • NT Support strongly suggests using an Indicator partial class instead
  • I.e. normal inheritance (an abstract base class) is not supported although some developers believe it works well
• Providing a constructor for an indicator is looking like a bad idea.
  • As this thread shows, doing so gets into problematic issues that are poorly understood.
  • It may be best to also deprecate indicator constructors.

The problem is that NT preempts the indicator's constructor with one of its own. How it then handles the indicator's constructor is problematic. Unless some new information turns up that makes this a bad idea, here is my suggestion:

• Deprecate constructors for indicators
  • The constructor NT provides will do basic construction, including initializing the Indicator base class.
  • I realize this is one more standard OOP thing for indicators to avoid.
  • Question: will this cause a problem? I have noticed that indicators the wizard produces have no constructor.
• NT must guarantee what will be called first.
  • Perhaps OnStateChange(State=SetDefaults)?
  • Knowing the initial entry point is crucial so that initialization can be performed.
  • So what is the guaranteed initial entry point?
• Indicators use that guaranteed entry point to do the initialization they would normally have done in a constructor.
  • If the entry is repeatedly called, such as OnStateChange(), It will be up to the indicator to ensure it does initialization only once.
  • Presumably, any such initialization should be very light, but that is already true for a constructor.

[-=Edge=-]

I just need to understand what is happening so that I can do whatever I need to do.

These won't help w/constructor question, but might help shed light w/OnStateChange()..

https://ninjatrader.com//support/helpGuides/nt8/NT%20HelpGuide%20English.html?onstatechange.htm

https://ninjatrader.com//support/helpGuides/nt8/NT%20HelpGuide%20English.html?understanding_the_li fecycle_of.htm

https://ninjatrader.com//support/helpGuides/nt8/NT%20HelpGuide%20English.html?ninjascript_best_pra ctices.htm

Be Safe in this Crazy World

[bltdavid]

I ran your test script and I see what you mean.

Unfortunately, I don't have any definitive answers.

I would recommend you avoid use of a constructor in
your indicators and use State.Configure for any one-time
initialization. Seems like you're thinking the same thing.

The Configure state is only seen by the indicator instance
actually running on the chart, ie, the instance that actually
sees and processes the bars.

Make sure you have a boolean 'ConfigureWasRan' so that
in State.Terminated you can release resources only if they
were actually initialized from State.Configure.

Just be aware that SetDefaults and Terminated will be called
a lot in your indicator. They are always paired, any time any
instance is sent SetDefaults, you are guaranteed that instance
will also receive Terminated.

Also, be aware, just the simple act of opening up the Indicators
dialog on any chart creates an instance of every indicator in
your system and runs SetDefaults for all of those indicator
instances. (This instance will eventually receive Terminated,
but this specific instance sees no other states -- but like I said,
SetDefaults/Terminated is always paired, even if no other
states are ever called.)

If you click on the indicator in the dialog to cause the properties
to be displayed, then this first instance is cloned and is sent
SetDefaults again. If you actually add it and click Ok, this
second instance is cloned yet again and SetDefaults is run
in the newly cloned instance. It is this third instance that
will continue running on the chart and see the bars.

At least, that's my understanding. I'm skipping over some
details, I'm sure.

-=o=-

Note that the restoration of a workspace's chart windows
might cause a different sequence of instances/states to
occur when compared to adding an indicator instance via
the Indicators dialog.

Duplicating a chart tab might be yet another different (or
same) sequence of events, I dunno.

Finally, whether the indicator dialog creates a new instance or
clones an existing cached instance (probably created and
cached at startup, or created/cached at end of last compile),
that's a good question. This deep internal cache (if there is
one), combined with perpetual cloning, might help explain the
weirdness you see with your constructor, but that's just a
guess. I don't know if the NinjaScript base.Clone() method
causes the constructor to execute (actually, from my own
testing, it looks like it does, but my gut says it shouldn't).

Unfortunately, I don't have a unifying theory as to what
is really happening. I mean, I don't know where or how,
exactly, the constructor fits into the overall sequence of
events.

It's a bit of a conundrum, thus I'd say State.Configure
seems like the best place to do (most) one-time
instance-based initialization -- since it only that
instance that is about to see the bars and do
anything really useful.

-=o=-

Thanks for the test script. I modified it some and have
attached the script I ended up playing with.

A lot more testing could be done, but I'm curious to hear
back from Chelsea -- I've never utilized the constructor
inside my indicators (at least not so deeply that I've
ever cared about it much), so I'm discovering these
new interesting NinjaScript tidbits as well.

[ETFVoyageur]

I fully appreciate that construction, SetDefaults, and terminate are run a LOT and, therefore, must be as light as possible. My situation is that I need a couple of very light initializations in place when SetDefaults is run. Any heavier initialization can (and should) wait for Configure (or later).

I'm trying to understand how things, like compile time, construction, SetDefaults, and deserialization, relate to each other. One example is that a way to lighten SetDefaults would be to have any appropriate values initialized at compile time. I've never seen anyone suggest that, so I presume there is some problem with doing so. What is the problem?

That's an example of why I want to understand just how and when data takes on its blend of compile time values, dynamic SetDefaults, deserialization, data passed in as parameters to the NT-generated constructor, etc. In other words, it would help when writing some indicators to know just what goes on from construction through SetValues, including the behind-the-scenes ordering. I have not seen that documented anywhere.

The more I think of it, the more I think that having a constructor coded in the indicator is a bad idea. Not only is the calling order just plain wrong, as my script demonstrates, but I also wonder whether the Indicator base class is getting initialized twice, perhaps in conflicting ways. The issue is constructor chaining. Surely the constructor doing the actual construction is chaining to the Indicator constructor. But so is the one I wrote in the indicator, which gets (belatedly) run. What happens when that second chaining call is made? Is the result what one would want?

As I posted previously, I am coming to the conclusion that writing a constructor into an indicator should be deprecated. Then do any light initialization at the start of SetDefaults and anything very intensive in Configure (or later).

What I need to know is what indicator code is guaranteed to be the first code run. Is that SetDefaults, or is it something else? Knowing a guaranteed starting point is crucial for properly initializing the indicator. Normally, that job falls to the constructor, but we are discussing doing away with constructors provided by the indicator.

There is one more reason to deprecate constructors -- we have to live with the system as it is today. The constructor written into the indicator is called out of order and I worry about what its effect on the base class (Indicator) might be. That's why I'm inclined to skip the whole problem rather than depend on something that seems to be poorly understood and, in any case, is implementation-dependent (i.e. subject to undocumented change).

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

For an indicator specifically, a new instance will start with the State.SetDefaults State. Once the strategy is constructed it will re-enable with the State.Configure State.

In general the Constructor is only used for VendorLicensing(). Any object instantiation or variable initialization would be from State.DataLoaded in OnStateChange().