Visual Studio and Magic Code

[ETFVoyageur]

Both a recent suggestion in this forum, and the online documentation in some places, suggest compiling in Visual Studio. That has the nice benefit of access to C# 6.0 now and (presumably) C# 7.0 when VS 15 comes out. I, among others, also find it a more convenient work flow.

Compiling in NT regenerates the Magic Code each time. Compiling in Visual Studio does not generate or regenerate the Magic Code. That means the magic code will get out of date for existing indicators, and will never get generated for new indicators.

What is the impact of that? How significant is it?

Thanks,
EV

[NinjaTrader_ChelseaB]

Hello EV,

The NinjaTrader generated code is to create overloads for calling your indicator.

If there are changes to a script that affect the properties always compile those changes using the NinjaScript Editor as NT will not be able to modify the generated code if the file is saved outside of the NinjaScript Editor.

This will affect the overloads of the method stubs for calling the indicator and will likely cause errors.

[ETFVoyageur]

I'm not clear on what you are saying. Here is what I notice:

• Use of Visual Studio is suggested at places in the online documentation -- not only running it from the NT editor, but also setting up working and running from VS. Working that way is pretty nice.
• The point of working in Visual Studio is to be able to make changes and compile there.
• Needing to be aware of what sort of changes require regenerating the Magic Code is a substantial burden, one that is likely to be overlooked.
• Even if one does remember, it is not enough to compile in the NT editor. My brief experimenting says the NT editor will not regenerate the Magic Code unless a change has been made by using the editor. Any change at all in the editor will trigger regenerating the Magic Code. No amount of changing outside the editor will code the Magic Code to be regenerated.

What is Best Practices when developing in VS to ensure that the Magic Code always gets regenerated when needed?

--EV

[NinjaTrader_ChelseaB]

Hello EV,

If you are using Visual Studio, anytime you make a change to a public property, compile using that script using the NinjaScript Editor.

[ETFVoyageur]

Note that recompiling alone will not help. You will have to bring the affected file up in the editor, make some change, and then compile. Otherwise the Magic Code will not get regenerated. That is manageable as a temporary workaround, but in the long run NT really needs a better way to handle this.

The current way is far too subject to human error. Furthermore, there is no practical way to ensure that all Magic Code is correct. Suppose I have been working on a number of files and I want to ensure that all of the Magic Code is correct -- that I did not ever forget to do a needed NT compile. The only current way would be to bring up each file in the project, make some modification to the file, and recompile. That is just plain unreasonable.

There needs to be SOME way to ensure that all Magic Code is correct, even if it is a brute-force as saying "rebuild all Magic Code". What would be much better would be for NT to keep track of the most recent date last modified it is aware of. Then it would know to regenerate Magic Code if changes happen outside the editor.

--EV

[ETFVoyageur]

As an example of something that would be practical for developers -- reliable, automatic, and not affecting the workflow -- how about providing something that could be run as a Visual Studio pre-build activity to update all Magic Code in the project?

The best way would be to update only files that had changed. Tracking that would not be hard -- just touch a flag file after doing any needed Magic Code updates. Then, the next time this is run, any file with a DLM more recent that the flag file needs its Magic Code updated.

--EV

[ETFVoyageur]

Something else occurs to me about the Magic Code -- does it really matter whether or not it is up to date? At the risk of sounding heretical, I doubt that it does.

Consider the facts (working in VS, so Magic Code is not automatically regenerated):

1) Compiling once in NT will get the original Magic Code generated. To the extent the caching helps performance, doing this would be a good idea. It also provides an expected interface. So, if you create a new indicator in VS by all means compile it in NT to get the original Magic Code. That's pretty easy to remember to do.

2) If you remove a public property from the indicator, the Magic Code will no longer compile, so you will be forced to do something about that. There is no chance of human error overlooking that. You can either recompile in NT, or else edit the code directly in VS (not hard to do).

3) If you add a new public property, the existing Magic Code will not know about it -- but so what? What harm will be done?

That last point requires a little amplification -- on looking at the Magic Code again (because of this discussion) I realized that the Magic Code already includes only some of the indicator's public properties. For example, it fails to include any public properties from an indicator's base class. It also fails to include any public properties that are in an expandable property the indicator uses, even if the expandable property class is defined in the same source file. So if it already fails to include all of the indicator's public properties, then what harm is done by failing include any new ones you add to the indicator?

There would be a minor convenience difference for the code that calls this function, and the caller could overlook the missing property. No performance difference, though -- it's an assignment either way. The real difference is timing. Ones the Magic Code knows about are used in an initializer list, and are applied prior to State.Configure. Ones the caller has to assign explicitly will happen after the indicator has already reached State.Configure. That is a logical problem, because all user configuration is supposed to be done before State.Configure. It is not a new problem, though, because (as noted above) Magic Code already fails to include some public properties.

(NT folks: is it a bug that Magic Code generation fails to include some public properties, resulting in the timing issue described above?)

I also note that it would not be difficult to use VS edit the Magic Code to include the new public property, if you want to do that. You could even edit in the ones that Magic Code generation misses, if that makes you happy. Just be aware that all of your edits will be gone the first time you compile in NT and it regenerates the Magic Code.

=====

That said, it would still be a good idea if NT were to provide something that could be used in the Visual Studio pre-build step to regenerate the Magic Code as needed. Doing so should not be difficult. (NT folks: please request that this be done.)

--EV

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

I have submitted a feature request on your behalf to regenerate the NinjaScript generated code when a script is saved outside of NinjaTrader (in Visual Studio).

Once I have a tracking ID for this request I will post this in this thread for future reference.

[ETFVoyageur]

Hello ETFVoyageur,

I have submitted a feature request on your behalf to regenerate the NinjaScript generated code when a script is saved outside of NinjaTrader (in Visual Studio).

Once I have a tracking ID for this request I will post this in this thread for future reference.

I could interpret that wording to mean the request is that NinjaTrader be aware when changes are made outside NinjaTrader so that NinjaTrader would regenerate the Magic Code the next time a compilation is done in NT. That's a good idea, should be quite easy to implement, and is one that I would support. This will be especially helpful for those who debug with Visual Studio by by clicking on the NT editor button and then attaching VS to a running NT.

The thing I specifically asked you to request, though, is a little different and is more important to those developing directly in Visual Studio. I am asking that NT provide something that can be run as a Visual Studio pre-build step to generate any needed Magic Code before the VS compilation takes place. It would be configured in the VS project "Pre-build event command line", just as the xcopy commands are currently configured in the "Post-build event command line" setting. This will be especially helpful for those who develop primarily in VS, and have VS start NT.

Your request may have already covered both, but I thought I should be clear that there are two separate requests to be made. The second one is the much more important one for those of us who develop primarily directly in VS.

--EV

[ETFVoyageur]

Chelsea,

Any comment on whether the fact that Magic Code generation fails to include certain public properties is a bug, or whether that is a deliberate design?

I do not see that it does any harm other than (a) altering the timing of setting the properties that Magic Code fails to support and (b) making those properties less discoverable.

--EV

[NinjaTrader_ChelseaB]

Hi EV,

Using as few words as possible, can you clarify what you would like to request?

You would like a pre-build command line that when executed in the command prompt will cause Visual Studio to create the NinjaScript generated code, is this correct?


I am not aware that NinjaTrader will not create an overload for specific object types when that particular script is compiled with the NinjaScript editor (and not with Visual Studio).
What object type are you referring to?
Do you have [NinjaScriptProperty] as an attribute of that object?

Can you give me an example (a snippet of a property not an entire script) that if I add to a script and compile with the NinjaScript Editor, NinjaTrader will not include with the overloads in the NinjaScript generate code?

[ETFVoyageur]

Hi EV,

Using as few words as possible, can you clarify what you would like to request?

You would like a pre-build command line that when executed in the command prompt will cause Visual Studio to create the NinjaScript generated code, is this correct?

Sort of -- I want an executable supplied by NinjaTrader and run from Visual Studio to create the Magic Code.

In VS, when you bring up the project properties and go to the "Build Events" settings you can configure this. People developing this way already have a couple of actions configured in the "Post-build event command line" box -- to xcopy the generated DLL and the PDB files. What I am asking for is something to be configured in the "Pre-build event command line" box that would generate any needed Magic Code.

I envision having an executable that would compare the modify date for each source file to the last time the executable had been run. Any files that were newer would be processed to see whether Magic Code generation is appropriate, and if so regenerating the Magic Code for that file. It would be easy to track the last time the executable ran by just keeping a flag file that the executable touches once it is done updating Magic Code.

The reason for this request is that it would automate regenerating the Magic Code in VS the same way it is already automated for those developing entirely in NT. That is obviously convenient, but far more important is the fact that having up to date (i.e. correct) Magic Code would no longer depend on the human noticing / remembering that he needs to take an action.

I am not aware that NinjaTrader will not create an overload for specific object types when that particular script is compiled with the NinjaScript editor (and not with Visual Studio).
What object type are you referring to?
Do you have [NinjaScriptProperty] as an attribute of that object?

Can you give me an example (a snippet of a property not an entire script) that if I add to a script and compile with the NinjaScript Editor, NinjaTrader will not include with the overloads in the NinjaScript generate code?

Let's talk about just one of the cases for now. Magic Code generation fails to correctly handle expandable properties. Such properties are themselves classes, with their own public properties. I had overlooked [NinjaScriptProperty] because I was not paying much attention to the Magic Code until this discussion. So I just added that property to both the expandable properties and to one of the properties (Enable) of each expandable class.

Here is the relevant Magic Code that NT generates. As you can see, the code is clearly incorrect.

    public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase
    {
        private VsaChartData[] cacheVsaChartData;
        public VsaChartData VsaChartData(BarNumbering barNumbers, FullData cursorData, FullData topData, bool enable, bool enable4)
        {
            return VsaChartData(Input, barNumbers, cursorData, topData, enable, enable);
        }

        public VsaChartData VsaChartData(ISeries<double> input, BarNumbering barNumbers, FullData cursorData, FullData topData, bool enable, bool enable4)
        {
            if (cacheVsaChartData != null)
                for (int idx = 0; idx < cacheVsaChartData.Length; idx++)
                    if (cacheVsaChartData[idx] != null && cacheVsaChartData[idx].BarNumbers == barNumbers && cacheVsaChartData[idx].CursorData == cursorData && cacheVsaChartData[idx].TopData == topData && cacheVsaChartData[idx].Enable == enable && cacheVsaChartData[idx].Enable == enable && cacheVsaChartData[idx].EqualsInput(input))
                        return cacheVsaChartData[idx];
            return CacheIndicator<VsaChartData>(new VsaChartData(){ BarNumbers = barNumbers, CursorData = cursorData, TopData = topData, Enable = enable, Enable = enable }, input, ref cacheVsaChartData);
        }
    }

Not only is the generated code incorrect, but it will not even compile. Compiling it generates a ton of error messages.

Severity    Code    Line    Description    File    Project    Suppression State
Error    CS0246    1780    The type or namespace name 'FullData' could not be found (are you missing a using directive or an assembly reference?)    C:\Users\Bob\Documents\NinjaTrader 8\bin\Custom\Indicators\VsaChartData.cs    NinjaTrader.Custom    Active
Error    CS0246    1739    The type or namespace name 'BarNumbering' could not be found (are you missing a using directive or an assembly reference?)    C:\Users\Bob\Documents\NinjaTrader 8\bin\Custom\Indicators\VsaChartData.cs    NinjaTrader.Custom    Active
Error    CS0246    1739    The type or namespace name 'FullData' could not be found (are you missing a using directive or an assembly reference?)    C:\Users\Bob\Documents\NinjaTrader 8\bin\Custom\Indicators\VsaChartData.cs    NinjaTrader.Custom    Active

<Many more errors, including syntax errors, deleted due to posting length restriction>

The ChartData indicator I sent you has expandable properties. If you add [NinjaScriptProperty] attributes as needed you should see the above results.

---EV

[ETFVoyageur]

I just simplified matters a bit so it would be easier to understand

• I moved the expandable object class out of the indicator. That gets into the well-known problem with Magic Code working best if you are willing to pollute the Indicators namespace. I did so for now to make things simpler. Philosophically, that is a terrible idea. (Yes, I am aware of the kludge-around.)
• I added an otherwise-unnecessary "using NinjaTrader.NinjaScript.Indicators;" so the Magic Code can see the classes.

The Magic Code that gets generated is still wrong. Here is the relevant part of the Magic Code that gets generated:

        public VsaChartData VsaChartData(ISeries<double> input, BarNumbering barNumbers, bool enable)
        {
            if (cacheVsaChartData != null)
                for (int idx = 0; idx < cacheVsaChartData.Length; idx++)
                    if (cacheVsaChartData[idx] != null && cacheVsaChartData[idx].BarNumbers == barNumbers && cacheVsaChartData[idx].Enable == enable && cacheVsaChartData[idx].EqualsInput(input))
                        return cacheVsaChartData[idx];
            return CacheIndicator<VsaChartData>(new VsaChartData(){ BarNumbers = barNumbers, Enable = enable }, input, ref cacheVsaChartData);
        }

Note that the code misuses "Enable", as if the indicator had such a property. The indicator has no such property, but the expandable property does. Here are the compiler errors that get generated:

Severity    Code    Line    Description    File    Project    Suppression State
Error    CS1061    1743    'VsaChartData' does not contain a definition for 'Enable' and no extension method 'Enable' accepting a first argument of type 'VsaChartData' could be found (are you missing a using directive or an assembly reference?)    C:\Users\Bob\Documents\NinjaTrader 8\bin\Custom\Indicators\VsaChartData.cs    NinjaTrader.Custom    Active
Error    CS0117    1745    'VsaChartData' does not contain a definition for 'Enable'    C:\Users\Bob\Documents\NinjaTrader 8\bin\Custom\Indicators\VsaChartData.cs    NinjaTrader.Custom    Active

Furthermore, I only tagged one expandable property and I have several If I tag them all, then Magic Code makes no distinction between the "Enable" from one of them and the "Enable" property from another of them.

Magic Code clearly has trouble with expandable properties. I know of no way to set up my code (other than avoiding expandable properties) that will show the properties in Magic Code and also get correct Magic Code generated. I'm going back to not using [NinjaScriptProperty] in these cases. At least then correct Magic Code will get generated, even if it does not include those properties.

--EV

[ETFVoyageur]

FWIW: The namespace issues I mentioned are describe by the esteemed kogonam in this post. They do not appear to have been fixed, even in NT8. This is clearly clumsy, but just as clearly it has not bothered people enough to make fixing it a priority.

--EV

[NinjaTrader_Brett]

Thanks for feedback and correct, the indicator generated code does not support expandable properties and only handles basic types. I'll add to our feedback list to potentially look into support in the future.