Announcement

Collapse
No announcement yet.

Partner 728x90

Collapse

[FYI] backtest solution "additional data series dynamically in an unsupported manner"

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [FYI] backtest solution "additional data series dynamically in an unsupported manner"

    Hi,

    for solving the error for additional data series that are configures with optimizer UI parameters the following code bypasses the problem, all that is necessary is to set the properties once. The new solution no longer wastes the setup period, it works instantly. The explanation why this is necessary is below. Use at your own risk and no warranty.

    Code:
        public static class StrategyOptimizationParameterInitializer {
            public static void InitializeParameters(Strategy strategy, string[] parameterNames) {
                bool isSuccessful = true;
                foreach(var parameterName in parameterNames) {
                    bool hasSetValue = false;
                    foreach(var parameter in strategy.OptimizationParameters) {
                        if(!parameter.Name.Equals(parameterName)) {
                            continue;
                        }
    
                        if(!parameter.Min.Equals(parameter.Max) || parameter.EnumValues.Length > 1) {
                            strategy.Print("Parameter: " + parameterName + " is set to more than one value, it will not be initialized");
                        } else {
                            object value = parameter.EnumValues.Length > 0 ? parameter.EnumValues[0] : parameter.Min;
                            System.Reflection.PropertyInfo propertyInfo = strategy.GetType().GetProperty(parameterName);
                            propertyInfo.SetValue(strategy, value, null);
                            hasSetValue = true;
                        }
                        break;
                    }
    
                    if(!hasSetValue) {
                        strategy.Print("Parameter: " + parameterName + " was not set, check if it's name is correct or values are set properly");
                        isSuccessful = false;
                    }
                }
    
                if(!isSuccessful) {
                    throw new Exception("Failed to initialize strategy parameters");
                }
            }
        }
    Strategy code example with parameter named Period, pass the name of the variable in an array to the initialization method.

    Code:
                } else if (State == State.Configure) {
                    if(Category == Category.Optimize || Category == Category.MultiObjective || Category == Category.WalkForward) {
                        if(Optimizer.NumberOfIterations > 0) {
                            StrategyOptimizationParameterInitializer.InitializeParameters(this, new string[]{"Period"});
                        }
                    }
    And that's it.

    If something goes wrong the strategy won't run but will let the user know in the NT output window.

    The workaround does not allow for optimizing the parameter, it has to be constant throughout the run.
    The problem is the strategy analyzer starts a dry run and doesn't initialize parameters and as a consequence the error occurs when the real testing starts.

    Outdated solution, no longer applies:
    Also for tests with a lot of data it is faster to recompile the strategy with constant values instead of this solution, because at least 1 period where NT prepares the data for the tests will be wasted.

    Code:
    namespace NinjaTrader.NinjaScript.Strategies {
        public class SomeStrategy : Strategy {
            private StaticSettings cachedSettings = StaticSettings.Instance();
    
            protected class StaticSettings {
                public bool Initialized = false;
                public int Period;
                private static StaticSettings instance = null;
    
                private StaticSettings() {
                }
    
                public static StaticSettings Instance() {
                    if(instance == null) {
                        instance = new StaticSettings();
                    }
                    return instance;
                }
            }
    
    ...
    
                } else if (State == State.Configure) {
                    int settingsPeriod = Period;
                    if(Category == Category.Optimize) {
                        bool isDryRun = Optimizer.NumberOfIterations > 0;
                        lock(cachedSettings) {
                            if(!isDryRun){
                                cachedSettings.Period = Period;
                                cachedSettings.Initialized = true;
                            } else if(cachedSettings.Initialized) {
                                settingsPeriod = cachedSettings.Period;
                            }
                        }
                    }
    
                   AddDataSeries(BarsPeriodType.Minute, settingsPeriod);
    Only applies to old solution:

    Back testing procedure:
    1. Set dynamic property e.g. Period
    4. Run test until failure
    5. Run test again


    How it works:
    First a dry run launches to create the bars for testing, but NT does not make the strategy analyzer settings available to it, so the dynamic data series parameter will be different.
    Then the real testing starts and iterations that are not the dry will copy their settings a static class. Unfortunately an exception occurs and the testing ends.
    On the restart the dry run will use the values from the cache that are the same as the settings from the strategy analyzer.

    It would be better to directly load the properties on the first run, but unfortunately if it is possible, I've not discovered how to do so.

    Another caveat is that once something is recompiled the static settings cache is lost and the procedure has to be restarted.

    Edit:
    simplified code & logic
    Edit X:
    added

    Code:
                } else if (State == State.Terminated) {
                    SettingsInitialized = false;
                }
    for getting templates to work.

    Edit X+1:
    removed SettingsInitialized

    Edit X+2:
    changed to dynamically setting properties
    Last edited by MojoJojo; 05-17-2020, 03:33 AM.

    #2
    Just bumping the thread because of the code update in case anybody finds it useful.

    Comment


      #3
      Unfortunately it seems like this workaround can on fresh optimizer tests and not with templates.

      After some more testing, it turns out that when saving optimizer templates suddenly SettingsInitialized is set to true on the first iteration, just like in the UI and thereby nullifying my workaround.
      I theorized that the first iteration would be a dry run but in light of this new information it looks more like a NT bug when passing optimizer parameters to a strategy.

      If it is the latter, I am kindly requesting the NT team to take a closer look, while this would affect only one sample, it's still an error.

      Comment


        #4
        Hello MojoJojo,

        While we appreciate the input you are providing on our forums to workaround unsupported items, the items are still unsupported and we would not be able to provide further assistance with those approaches.

        If you are presenting a potential bug, could you demonstrate with a simplified use case that is not intended to workaround something which is unsupported?

        Note that support for dynamically adding data series is being tracked with the internal feature request ticket SFT-882. I have added a vote on your behalf. For anyone else that would like to have their vote tracked, please let us know.

        Feature Request Disclaimer

        We receive many requests and cannot reasonably implement all requested features or changes. Interest is tracked internally and if enough interest is tracked, it would be weighed against how feasible it would be to make those changes to consider implementing.

        When new features are implemented, they will be listed in the Release Notes page of the Help Guide. The ID number will be different than the internal feature request tracking ID, but the description of the feature will let you know if that feature has been implemented.

        Release Notes -
        https://ninjatrader.com/support/help...ease_notes.htm

        I look forward to assisting.

        Comment


          #5
          Hello Jim,

          thanks for your response, optimizing dymanically added data series would be nice. I would be happy if during optimization data series with a constant parameter from the ui would function which is what my workaround tries to achieve.

          Here is a code sample and the output to substantiate my claims about the optimizer passing parameters to a strategy instance.

          Code:
              public class FirstIterationWrongStrategy : Strategy {
                  private class IterationCounter {
                      private static IterationCounter instance = null;
                      public int Iteration = 0;
                      public static IterationCounter GetInstance() {
                          if(instance == null) {
                              instance = new IterationCounter();
                          }
                          return instance;
                      }
                  }
                  private IterationCounter iterationCounter = IterationCounter.GetInstance();
                  protected override void OnStateChange() {
                      if (State == State.SetDefaults) {
                          Description                                    = @"";
                          Name                                        = "FirstIterationWrongStrategy";
                          Calculate                                    = Calculate.OnBarClose;
                          EntriesPerDirection                            = 1;
                          EntryHandling                                = EntryHandling.AllEntries;
                          IsExitOnSessionCloseStrategy                = true;
                          ExitOnSessionCloseSeconds                    = 30;
                          IsFillLimitOnTouch                            = false;
                          MaximumBarsLookBack                            = MaximumBarsLookBack.TwoHundredFiftySix;
                          OrderFillResolution                            = OrderFillResolution.Standard;
                          Slippage                                    = 0;
                          StartBehavior                                = StartBehavior.WaitUntilFlat;
                          TimeInForce                                    = TimeInForce.Gtc;
                          TraceOrders                                    = false;
                          RealtimeErrorHandling                        = RealtimeErrorHandling.StopCancelClose;
                          StopTargetHandling                            = StopTargetHandling.PerEntryExecution;
                          BarsRequiredToTrade                            = 20;
                          // Disable this property for performance gains in Strategy Analyzer optimizations
                          // See the Help Guide for additional information
                          IsInstantiatedOnEachOptimizationIteration    = true;
                          Period = 30;
                          SettingsInitialized = false;
          
                      } else if (State == State.Configure) {
                          lock(iterationCounter) {
                              Print(String.Format("Iteration: {0} Period: {1} Initialized: {2}", new object[]{iterationCounter.Iteration++, Period, SettingsInitialized} ));
                          }
                      }
                  }
          
                  protected override void OnBarUpdate() {
                  }
          
                  #region Properties
                  [NinjaScriptProperty]
                  [Range(1, int.MaxValue)]
                  [Display(Name="Period", Order=1, GroupName="Parameters")]
                  public int Period
                  { get; set; }
          
                  [NinjaScriptProperty]
                  [Display(Name="Settings Intitialized", Order=28, GroupName="Parameters")]
                  public bool SettingsInitialized
                  { get; set; }
                  #endregion
              }
          The UI input is always ignored on the first iteration until I save a template. Hope that helps with a fix if one is needed

          Addendum:
          When a template is saved, NT will keep the values of the last strategy iteration it has run instead of the defaults. On the next optimization it will use these values on the first iteration until another template is saved.
          Selecting one result of the optimizer output counts as an iteration. The unfortunate part is that when an optimization fails, there is no output, so it cannot be used as a workaround.

          It's still possible the first iteration is a dry run, because more iterations are executed than appear to be necessary.
          My assumition is: minimum executed iterations = dry run + test iterations + UI first result output rendering.
          Attached Files
          Last edited by MojoJojo; 04-29-2020, 12:09 PM.

          Comment


            #6
            Armed with new knowledge, the workaround can also be used with templates now

            Code:
                        } else if (State == State.Terminated) {
                            SettingsInitialized = false;
                        }

            Comment


              #7
              Hello MojoJojo,

              It sounds like you have been able to move forward, but I should be clear for the sake of the forum that this approach would not be something that we would officially support.

              If there is another topic we can assist with, please don't hesitate to open a new thread.

              Comment


                #8
                The code and procedure has been simplified thanks to being able to tell from strategy variables when the dry run occurs and therefore the eliminaton of the property SettingsInitialized.

                Comment


                  #9
                  The last version works instantly and is much cleaner. And that should be it, so long and thanks for all the fish.

                  Comment


                    #10
                    I would definitely like to request that internal feature request ticket SFT-882 be implemented. Dynamically added data series could change the utility of
                    NT strategies by an exponential factor if they're studying accurate code. Its hard to believe that this hasn't been done as it could bring the platform back
                    from the scrapheap..

                    Comment


                      #11
                      +1

                      Please add my vote.

                      Comment


                        #12
                        Hello Chuck_63 and bltdavid,

                        I've added your votes to SFT-882.

                        Thank you for providing a voice on this request.
                        Chelsea B.NinjaTrader Customer Service

                        Comment


                          #13
                          +1
                          Please add my vote too.
                          eDanny
                          NinjaTrader Ecosystem Vendor - Integrity Traders

                          Comment


                            #14
                            Hello eDanny,

                            I've added your vote to SFT-882 as well.
                            Thank you for your voice on this request.
                            Chelsea B.NinjaTrader Customer Service

                            Comment

                            Latest Posts

                            Collapse

                            Topics Statistics Last Post
                            Started by Geovanny Suaza, 02-11-2026, 06:32 PM
                            0 responses
                            648 views
                            0 likes
                            Last Post Geovanny Suaza  
                            Started by Geovanny Suaza, 02-11-2026, 05:51 PM
                            0 responses
                            369 views
                            1 like
                            Last Post Geovanny Suaza  
                            Started by Mindset, 02-09-2026, 11:44 AM
                            0 responses
                            108 views
                            0 likes
                            Last Post Mindset
                            by Mindset
                             
                            Started by Geovanny Suaza, 02-02-2026, 12:30 PM
                            0 responses
                            572 views
                            1 like
                            Last Post Geovanny Suaza  
                            Started by RFrosty, 01-28-2026, 06:49 PM
                            0 responses
                            574 views
                            1 like
                            Last Post RFrosty
                            by RFrosty
                             
                            Working...
                            X