Persisent state data

[ETFVoyageur]

I have seen other threads on this topic, but none of them really answer my issue; I'm also not sure the older ones match the current product.

My indicator adds a menu to the chart toolbar. The menu entries are used to toggle various things on and off. I use IsChecked so that the menu entries show whether that item is enabled.

NinjaTrader freely creates new objects, such as when connecting to a data source. That is fine, but I need the state of the menu entries to persist so that things do not change underneath the user when any of several things occur that cause a new instance to be created.

I have read that the way to persist data is to use public properties. As I understand it, there are two independent relevant attributes:
[Browsable(false)] -- true/false controls whether or not the property is shown in the configuration dialog.
[XmlIgnore] -- controls whether or not the property gets serialized (i.e. is persistent).

There must be something wrong with my understanding. The properties I want to persist are type bool so there should be no problem serializing them. I do not set [XmlIgnore] so I expect the property to be persisted. Unfortunately, if I set [Browsable(false)], to keep the property out of the configuration dialog, the property is not persisted. The property is persisted if I set no attributes, but then it also appears in the configuration dialog -- which I do not want.

So, the obvious question is: how can my indicator have persistent state variables without them showing in the configuration dialog?

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

What you do mean by persist? Do you mean the variable value is saved with xml workspaces and templates?

Do not use the [XmlIgnore()] attribute as you have suggested.
[Browsable(false)] is correct for keeping these out of the UI. Also, do not use the [NinjaScriptProperty] attribute to keep this out of the overloads.

Do you mean the script is reloaded? If so, you might need to set the value of the property to the clone.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/indicator-development/1224456-how-to-hide-unhide-a-plot#post1225348

[ETFVoyageur]

Chelsea,

Thanks for your suggestion. It works perfectly for me for the testing I have done. I have done a lot of searching for the answer and your suggestion is the first one I have found that addresses the problem without being some sort of hack. Here is my code, in case you can spot any misunderstanding I may have:

public override object Clone()
{

object clonedInstance = base.Clone();
VsaChartData obj = clonedInstance as VsaChartData;
if (obj == null) return clonedInstance;

// Update the non-GUI data that needs to be persistent.
obj.ShowTickerDescriptionEnabled = ShowTickerDescriptionEnabled;
obj.ShowBarNumbersEnabled = ShowBarNumbersEnabled;
obj.CursorShowBarNumberEnabled = CursorShowBarNumberEnabled;
obj.CursorShowBarDataEnabled = CursorShowBarDataEnabled;
obj.ToolbarDataEnabled = ToolbarDataEnabled;
return obj;

}

I have found the same issue/problem raised back for over 9 years. It seems to me that there is an underlying design issue (flaw?) that is causing the problem. Whether to show a property in the configuration dialog and whether to serialize it are logically two independent things, but the NinjaTrader design blurs them. IMHO there should be two independent attributes -- one for whether to include the property in the configuration dialog and another for whether to serialize the property.

Instead, the current situation is that a property is serialized if, and only if, it is shown in the configuration dialog (modulo [XmlIgnore]). That's unfortunate because it precludes serializing any state that is not appropriate for the configuration dialog. I, and based on my reading, a lot of other developers would be much happier if the two concepts were really independent. The system could default to the current behavior -- serialized if and only if it is in the dialog -- but it should be possible for the developer to override that.

[ETFVoyageur]

I do have one more thought about this solution. It does solve the operational issue -- an indicator can have a durable state during a single session. I am glad Chelsea brought it to our attention. However, since the state is not serialized it will not be preserved across sessions.

What we really need is for NinjaTrader to provide a way to handle configuration display and serialization independently. There needs to be one attribute that says to serialize or not, and has no other effect. There needs to be another attribute that says to display in the configuration dialog or not, and has no other effect.

Please take this request seriously. It would be easy to implement and it would fix something that has been bothering developers for at least the better part of a decade.

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

Are you referring to the end of session, as in the end of a trading day session defined in the Trading Hours template in the Sessions?

Or are you referring to reloading the script?

The help guide does state that only browsable properties will be cloned.

From the help guide:
" Cloning NinjaScript
Clone is the operation of iterating over all public browsable properties on a NinjaScript object and duplicating the values over to a freshly generated instance. For the majority of NinjaScript with standard properties the clone process is transparent to you and you do not need to be concerned the the clone process. For those of you that want more control or will be utilizing complex properties then knowledge about clone is essential."

https://ninjatrader.com/support/helpGuides/nt8/understanding_the_lifecycle_of.htm

"By default, the NinjaScript Clone() method will copy all the Property Info and Browsable Attributes to the new instance when the object is created"

https://ninjatrader.com/support/helpGuides/nt8/clone.htm

I will submit a feature request on your behalf for the development team to consider persisting properties using the Browsable(false) attribute to clones.

Once I have a tracking ID for this I will post in this thread for future reference.

[ETFVoyageur]

To summarize the situation:

1. The current system has no implementation bug – it is operating as designed and documented. There is, however, a design bug.
2. There are two independent concepts: whether a property should be included in the configuration dialog and whether the property should be serialized. That results in four possible combinations.
3. The current system handles only three of the combinations:
   1. Dialog & serialize – [Browsable(true)] public.
   2. Dialog & not serialize – [Browsable(true)] [XmlIgnore] public.
   3. Not dialog & not serialize – [Browsable(false)] or else not public.
4. The current system design fails to provide for the fourth possibility: not dialog & serialize.
   1. This has been a running complaint from developers for at least the better part of a decade.
   2. An indicator should be able to maintain serializable state that is not shown in the configuration dialog.

The inability to serialize properties that are not part of the configuration dialog is a design bug. Not an implementation bug -- a design bug. This is a serious shortcoming for indicators with state that should be serialized but that is inappropriate for the configuration dialog. Overriding Clone () helps but does not fully address the problem because that does not provide serialization.

[ETFVoyageur]

I still do not see how to handle the missing fourth case -- serializing a property that is not displayed in the indicator configuration dialog.

Is there any way for an indicator to augment NT's serialization so the indicator can serialize such a property?

Note: I still consider it a serious NT design bug but, be that as it may, I need a way to work with NT as it is.

[eDanny]

How about modifying properties using [PersistentDataAttribute(true)]? Make your properties disappear but lets them be serialized and saved.

[AttributeUsage(AttributeTargets.Property)]
public class PersistentDataAttribute : Attribute
{
public bool enabled { get; private set; }
// Default parameters / optional arguments were first supported
// in .NET 4.0; NinjaTrader 7 uses .NET 3.5
// Get the same effect by overloading the constructor
public PersistentDataAttribute ( ) { enabled = true; }
public PersistentDataAttribute ( bool s ) { enabled = s; }
}

protected virtual void ModifyProperties(PropertyDescriptorCollection col)
{
// Remove any properties with the [PersistentData] attribute
foreach (PropertyInfo prop in this.GetType().GetProperties().Where (prop => Attribute.IsDefined(prop, typeof(PersistentDataAttribute))))
{
Attribute[] attr = (Attribute[])prop.GetCustomAttributes(typeof(PersistentDataAtt ribute), false);
if ( (attr==null) || ( ! ((PersistentDataAttribute)attr[0]).enabled) ) return;
col.Remove(col.Find(prop.Name, true));
}
}

public AttributeCollection GetAttributes()
{
return TypeDescriptor.GetAttributes(GetType());
}

public string GetClassName()
{
return TypeDescriptor.GetClassName(GetType());
}

public string GetComponentName()
{
return TypeDescriptor.GetComponentName(GetType());
}

public TypeConverter GetConverter()
{
return TypeDescriptor.GetConverter(GetType());
}

public EventDescriptor GetDefaultEvent()
{
return TypeDescriptor.GetDefaultEvent(GetType());
}

public PropertyDescriptor GetDefaultProperty()
{
return TypeDescriptor.GetDefaultProperty(GetType());
}

public object GetEditor(Type editorBaseType)
{
return TypeDescriptor.GetEditor(GetType(), editorBaseType);
}

public EventDescriptorCollection GetEvents(Attribute[] attributes)
{
return TypeDescriptor.GetEvents(GetType(), attributes);
}

public EventDescriptorCollection GetEvents()
{
return TypeDescriptor.GetEvents(GetType());
}

public PropertyDescriptorCollection GetProperties(Attribute[] attributes)
{
PropertyDescriptorCollection orig = TypeDescriptor.GetProperties(GetType(), attributes);
PropertyDescriptor[] arr = new PropertyDescriptor[orig.Count];
orig.CopyTo(arr, 0);
PropertyDescriptorCollection col = new PropertyDescriptorCollection(arr);

ModifyProperties(col);
return col;

}

public PropertyDescriptorCollection GetProperties()
{
return TypeDescriptor.GetProperties(GetType());
}

public object GetPropertyOwner(PropertyDescriptor pd)
{
return this;
}

[Morkab]

If NinjaTrader's serialization mechanism doesn't behave as expected with the [Browsable(false)] attribute, you can implement custom serialization logic using the OnStateSerialize and OnStateDeserialize methods provided by NinjaTrader. Within these methods, you have full control over what CRM data enrichment gets serialized and how.
protected override void OnStateSerialize(BinaryWriter writer)
{
// Serialize your state variables here
writer.Write(_menuEntryEnabled);
}

protected override void OnStateDeserialize(BinaryReader reader)
{
// Deserialize your state variables here
_menuEntryEnabled = reader.ReadBoolean();
}


If you still want to use properties for simplicity but hide them from the configuration dialog, you might explore creating a custom configuration interface or settings panel within your indicator. This way, users can still configure the behavior of your indicator, but it won't clutter the default configuration dialog.

[ETFVoyageur]

Morkab,

Thanks. I like your suggestion's simplicity. My values are properties only in the hope of having NT serialize them. I'd prefer them to be simple booleans. Unfortunately, when I try your suggestion I get a compiler error: "no suitable method found to override". What am I missing?

public class VsaChartData : Indicator
{
        ...
        protected override void OnStateSerialize(BinaryWriter writer)
        {}
}

One thought occurs to me -- I am using System.IO.BinaryWriter. Are you using some other one?

[ETFVoyageur]

eDanny,

Thanks for the suggestion. I have exactly that code but something is not working for me -- the PersistentData value is not being set properly. All of the following have the same effect -- they act as if their value is false:

[PersistentData]
[PersistentData(true)]
[PersistentData(false)]

The attribute code always acts as if the attribute value is false:

[AttributeUsage(AttributeTargets.Property)]
public class PersistentDataAttribute : Attribute {
    public bool enabled { get; private set; }
    // Default parameters / optional arguments were first supported in .NET 4.0; NinjaTrader 7 uses .NET 3.5
    // Get the same effect by overloading the constructor
    public PersistentDataAttribute ( ) { enabled = true; }
    public PersistentDataAttribute ( bool s ) { enabled = s; }
}

I know the problem is with the attribute definition because I can make persistent data work by changing the non-default constructor to:

public PersistentDataAttribute ( bool s ) { enabled = true; }

Then all three cases work. I see two wrong things:

1) The non-default constructor is always being used, even when there is no argument. The change should not affect the no-argument case, but it does.

2) When there is an argument its value is being ignored.

Any ideas?

Thanks.

===== Later note =====
It looks to me as if the problem is that all instances take on the same value. I'm still investigating, but the behavior could be normal ... just not what I would expect. I would expect different instances to be independent, but they do not seem to be.

[ETFVoyageur]

I found the problem. The code eDanny posted matches the code I am porting -- and that code has a bug resulting in not processing any properties beyond the first one whose value is false. Here is the original code:

protected virtual void ModifyProperties(PropertyDescriptorCollection col)
{
     // Remove any properties with the [PersistentData] attribute
     foreach (PropertyInfo prop in this.GetType().GetProperties()
                   .Where (prop => Attribute.IsDefined(prop, typeof(PersistentDataAttribute))))
     {
          Attribute[] attr = (Attribute[])prop.GetCustomAttributes(typeof(PersistentDataAttribute), false);
          if ( (attr==null) || ( ! ((PersistentDataAttribute)attr[0]).enabled) ) return;
          col.Remove(col.Find(prop.Name, true));
     }
}​

The bad line is

if ( (attr==null) || ( ! ((PersistentDataAttribute)attr[0]).enabled) ) return;

That line should continue, not return!

if ( (attr==null) || ( ! ((PersistentDataAttribute)attr[0]).enabled) ) continue;

With that one bug fix the suggested code is fine and solves the original problem -- how to have a serialized property that does not appear in the GUI. Some indicators need that to serialize their state. The way to do so is to set a public property with two attributes:

[Browsable]           // so NinjaTrader will serialize it
[PersistentData]      // so it will not appear in the GUI

[eDanny]

BTW, the code works for me whether the line has 'continue' or 'return'.
Also, I think you gave this to me for NT7 years ago.

[ETFVoyageur]

The problem with "return" is that if you have several persistent properties it will stop processing after the first one that has [PersistentData(false)]. "Continue" will process all of the properties. You will never notice the difference unless you have one, other than the last one, set to false. If you just set all to [PersistentData], which would be a common usage, you will never encounter the problem.

Imay have given you the code, which would account for it being identical, but it was long enough ago that I do not recall.