Strategy Analyzer Optimization result table shows incorrect data

[wzgy0920]

Hello wzgy0920,

If you also want to reset variables on a new session, the logic in OnBarUpdate() would need to accomplish this. You can reset when Bars.IsFirstBarOfSession is true.
https://ninjatrader.com/support/help...rofsession.htm

An optimization will run a backtest (iteration) for each set of input parameters being optimized over.

For example with the SampleMACrossover using Fast as 1;5;2 and Slow 1;1;1 will optimize 3 backtest iterations. Fast 1 / Slow 1, Fast 3 / Slow 1, Fast 5 / Slow 1.

When IsInstantiatedOnEachOptimizationIteration is false, NinjaTrader will re-use generated strategy class objects for new backtest iterations and restart the historical data back at bar 0. When this is true, a new strategy class object is generated for every iteration which will use more memory and CPU resources.

Thank you for the insight. I have updated my strategy to ensure I reset my variables during state transitions to State.DataLoaded. In addition, I also reset my variables when Bars.IsFirstBarOfSession = true. Further more, 1 hour before session close (i.e. at 4pm EST) I also cancel all orders and exit all positions. This means that my strategy does not trade after 4pm EST and at the first bar of the session at 6pm EST, all variables are cleared and resetted.I am still have the issue I originally reported.

As particularly head scratching issue is that the only some of the values displayed in the rows of the top 10 results are incorrect. However, if I click on a specific result (i.e. specific parameter set) the values displayed in the Summary is correct. What is mean is that if I run a Backtest of that specific parameter set, the results shown in Summary will be the same if IsInstantiatedOnEachOptimizationIteration is true or false.

[NinjaTrader_ChelseaB]

Hello wzgy0920,

My thoughts would be there is some other code causing the behavior.

May I have a reduced export to test on my end?

To export a NinjaTrader 8 NinjaScript so this can be shared and imported by the recipient do the following:

1. Click Tools -> Export -> NinjaScript Add-on...
2. Click the 'add' link -> check the box(es) for the script(s) and reference(s) you want to include
3. Click the 'Export' button
4. Enter a unique name for the file in the value for 'File name:'
5. Choose a save location -> click Save
6. Click OK to clear the export location message

By default your exported file will be in the following location:

• (My) Documents/NinjaTrader 8/bin/Custom/ExportNinjaScript/<export_file_name.zip>

Below is a link to the help guide on Exporting NinjaScripts.
http://ninjatrader.com/support/helpG...nt8/export.htm

[wzgy0920]

When IsInstantiatedOnEachOptimizationIteration is false, NinjaTrader will re-use generated strategy class objects for new backtest iterations and restart the historical data back at bar 0. When this is true, a new strategy class object is generated for every iteration which will use more memory and CPU resources.

Is there a way I can calculate a unique ID for each instance of the strategy class object? I want to include the ID in my print statements so I can know which instance is printing the output.

[NinjaTrader_ChelseaB]

Hello wzgy0920,

Yes, this is demonstrated in the OptimizationReuseInstanceStatesTest I have provided you.

[wzgy0920]

Hello wzgy0920,

My thoughts would be there is some other code causing the behavior.

May I have a reduced export to test on my end?

Thank you for help so far. I was able to create a simplified example strategy that replicates the issue I am reporting. I have emailed the code to NT support email. I hope this is helps us troubleshoot the issue.

[wzgy0920]

After troubleshooting with ChelseaB via email, I have found the the issue. See details below. I have NT can fix this issue as soon as possible.

Summary: A major bug in NT 8 is causing the value of Bars.BarsSinceNewTradingDay to become negative when using Optimize with IsInstantiatedOnEachOptimizationIteration=false. When the instance of the strategy object is reused (i.e. second time or third time), the value of Bars.BarsSinceNewTradingDay is outputted as a negative value.

I believe this bug is related to post made by other users:

• https://forum.ninjatrader.com/forum/...timization-etc
• https://forum.ninjatrader.com/forum/...-false-problem

Code to reproduce the issue:

//
// Copyright (C) 2022, NinjaTrader LLC <www.ninjatrader.com>.
// NinjaTrader reserves the right to modify or overwrite this NinjaScript component with each release.
//
#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.Indicators;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds strategies in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Strategies
{    
    public class OptimizeIssueExample : Strategy
    {
        private SMA smaFast;
        private SMA smaSlow;

        private Guid                    instanceGUID    = Guid.NewGuid();
        private int DebugPrintLevel = 0;

        private int firstSessionBarIndex = -1;
        private int instanceRunCount = 0;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description    = "A simple example to replicate issue with Optimize.";
                Name        = "OptimizeIssueExample";
                Fast        = 10;
                Slow        = 25;
                // This strategy has been designed to take advantage of performance gains in Strategy Analyzer optimizations
                // See the Help Guide for additional information
                IsInstantiatedOnEachOptimizationIteration = false;
            }
            else if (State == State.DataLoaded)
            {
                smaFast = SMA(Fast);
                smaSlow = SMA(Slow);

                smaFast.Plots[0].Brush = Brushes.Goldenrod;
                smaSlow.Plots[0].Brush = Brushes.SeaGreen;

                AddChartIndicator(smaFast);
                AddChartIndicator(smaSlow);

                Print(string.Format("Strategy: {0} IsInstantiatedOnEachOptimizationIteration: {1} Fast: {2}  Slow: {3} State: {4} GUID: {5} instanceRunCount: {6}", Name, IsInstantiatedOnEachOptimizationIteration, Fast, Slow, State, instanceGUID, instanceRunCount));

                instanceRunCount += 1;
            }
        }

        protected override void OnBarUpdate()
        {

            if (Bars.IsFirstBarOfSession)
            {
                if (instanceRunCount >= 2)
                    PrintTo = PrintTo.OutputTab2;
                else
                    PrintTo = PrintTo.OutputTab1;                
                Print(string.Format("Time: {0} Bars.BarsSinceNewTradingDay: {1} CurrentBar: {2}  Fast: {3}  Slow: {4} GUID: {5} instanceRunCount: {6}", Times[0][0], Bars.BarsSinceNewTradingDay, CurrentBar, Fast, Slow, instanceGUID, instanceRunCount));
                PrintTo = PrintTo.OutputTab1;

            }

            if (CrossAbove(smaFast, smaSlow, 1))
                EnterLong();
            else if (CrossBelow(smaFast, smaSlow, 1))
                EnterShort();
        }


        #region Properties
        [Range(1, int.MaxValue), NinjaScriptProperty]
        [Display(ResourceType = typeof(Custom.Resource), Name = "Fast", GroupName = "NinjaScriptStrategyParameters", Order = 0)]
        public int Fast
        { get; set; }

        [Range(1, int.MaxValue), NinjaScriptProperty]
        [Display(ResourceType = typeof(Custom.Resource), Name = "Slow", GroupName = "NinjaScriptStrategyParameters", Order = 1)]
        public int Slow
        { get; set; }
        #endregion
    }
}
​

[Browniver]

Thanks for bringing up this issue. I have experienced the same thing systematically. Using the latest NT8 version. Have you guys found an explanation or solution? Here is a example:

[NinjaTrader_ChelseaB]

Hello Browniver,

The behavior with Bars.BarsSinceNewTradingDay was marked as corrected with ID 4530 in the release notes.

https://ninjatrader.com/support/helpGuides/nt8/8_1_2_1.htm

May I confirm you have updated to 8.1.2.1 and you are able to reproduce?

Are you printing Bars.BarsSinceNewTradingDay to the output window along with a instance id?

[Browniver]

May I confirm you have updated to 8.1.2.1 and you are able to reproduce? YES

Are you printing Bars.BarsSinceNewTradingDay to the output window along with a instance id? NO
Does this mean it does recalculate the strategy and thats why I get different results?

[NinjaTrader_ChelseaB]

Hello Browniver,

Just to confirm, you are following this thread and seeing the issue is with Bars.BarsSinceNewTradingDay and that is why you have posted in this thread because you are also having an issue with Bars.BarsSinceNewTradingDay?

How do you know this if you are not printing the value of Bars.BarsSinceNewTradingDay to the output window?

[Browniver]

Hi Chelsea

> Just to confirm, you are following this thread and seeing the issue is with Bars.BarsSinceNewTradingDay and that is why you have posted in this thread because you are also having an issue with Bars.BarsSinceNewTradingDay?

I was looking for an explanation why the Data in the overview table at the top does not correspond with the data in the summary table. It only corresponds directly after the optimization task later it starts recalculating the data and does never correspond. The data in the log on the buttom always corresponds with the summary table (in the middle) but not the data on the top (overview). I would like to know what happens here and why it is like that. Looking for an explanation I have found this thread seeing somebody else having a similar issue.

I dont know whether I have an issue with Bars.BarsSinceNewTradingDay or not. I just didnt see any Bars.BarsSinceNewTradingDay syntax in the code (or output window) created with strategy builder.

Can you explain why it is like this? I dont see the logic behind.

Thanks Chelsea!

[NinjaTrader_ChelseaB]

Hello Browniver,

The cause of the behavior with wzgy0920 was with Bars.BarsSinceNewTradingDay.

See post # 21.

If this is not being used in your script, you will need to identify what specific code is causing the behavior the same as wzgy0920 did by reducing and using prints.

[Browniver]

Do you have any hints what to look for? Any threads that may provide a guideline?

I read so many threads and a few address the same issue but these threads are either old or do not provide any solution.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.Indicators;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Strategies in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Strategies
{
    public class AAASLTPBESTRRV : Strategy
    {
        private double Initial_Stop_Long_Price;
        private double Initial_Stop_Short_Price;
        private double Breakeven_Trigger_Price_Store;
        private bool BreakevenBool;
        private bool STr_Trailing_Bool;
        private double STr_Trigger_Price_Store;

        private NinjaTrader.NinjaScript.Indicators.TradeSaber.TrendMagicTSmod TrendMagicTSmod1;
        private NinjaTrader.NinjaScript.Indicators.TradeSaber.TrendMagicTSmod TrendMagicTSmod2;
        private TSSuperTrend TSSuperTrend1;
        private TSSuperTrend TSSuperTrend2;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Enter the description for your new custom Strategy here.";
                Name                                        = "AAASLTPBESTRRV";
                Calculate                                    = Calculate.OnPriceChange;
                EntriesPerDirection                            = 9;
                EntryHandling                                = EntryHandling.AllEntries;
                IsExitOnSessionCloseStrategy                = true;
                ExitOnSessionCloseSeconds                    = 120;
                IsFillLimitOnTouch                            = false;
                MaximumBarsLookBack                            = MaximumBarsLookBack.TwoHundredFiftySix;
                OrderFillResolution                            = OrderFillResolution.Standard;
                Slippage                                    = 0;
                StartBehavior                                = StartBehavior.WaitUntilFlatSynchronizeAccount;
                TimeInForce                                    = TimeInForce.Gtc;
                TraceOrders                                    = true;
                RealtimeErrorHandling                        = RealtimeErrorHandling.StopCancelClose;
                StopTargetHandling                            = StopTargetHandling.PerEntryExecution;
                BarsRequiredToTrade                            = 20;
                // Disable this property for performance gains in Strategy Analyzer optimizations
                // See the Help Guide for additional information
                IsInstantiatedOnEachOptimizationIteration    = true;
                Position_Size                    = 1;
                Start_Time                        = DateTime.Parse("00:00", System.Globalization.CultureInfo.InvariantCulture);
                Stop_Time                        = DateTime.Parse("23:00", System.Globalization.CultureInfo.InvariantCulture);
                SL_Long                    = -50;
                SL_Short                    = 50;
                TP_Long                    = 50;
                TP_Short                    = -50;
                BE_Long                    = 25;
                BE_Short                    = -25;
                BE_Offset                    = 0;
                Trail_Trigger_Long                    = 30;
                Trail_Trigger_Short                    = -30;
                STr_Period                    = 12;
                STr_Multiplier                    = 1;
                STr_Smooth                    = 6;
                Trendmagic_Multi_Fast                    = 0.5;
                Trendmagic_Multi_Slow                    = 2;
                Initial_Stop_Long_Price                    = 0;
                Initial_Stop_Short_Price                    = 0;
                Breakeven_Trigger_Price_Store                    = 1;
                BreakevenBool                    = false;
                STr_Trailing_Bool                    = false;
                STr_Trigger_Price_Store                    = 1;
            }
            else if (State == State.Configure)
            {
            }
            else if (State == State.DataLoaded)
            {                
                TrendMagicTSmod1                = TrendMagicTSmod(Close, 20, 14, Trendmagic_Multi_Fast, false, false, 25, 25, false);
                TrendMagicTSmod2                = TrendMagicTSmod(Close, 20, 14, Trendmagic_Multi_Slow, false, false, 25, 25, false);
                TSSuperTrend1                = TSSuperTrend(Close, SuperTrendMode.ATR, Convert.ToInt32(STr_Period), STr_Multiplier, MovingAverageType.HMA, Convert.ToInt32(STr_Smooth), false, false, false);
                TSSuperTrend2                = TSSuperTrend(Close, SuperTrendMode.ATR, Convert.ToInt32(STr_Period), STr_Multiplier, MovingAverageType.HMA, Convert.ToInt32(STr_Smooth), false, false, false);
                TrendMagicTSmod1.Plots[0].Brush = Brushes.Orange;
                TrendMagicTSmod1.Plots[1].Brush = Brushes.Plum;
                TrendMagicTSmod2.Plots[0].Brush = Brushes.Orange;
                TrendMagicTSmod2.Plots[1].Brush = Brushes.DarkViolet;
                TSSuperTrend1.Plots[0].Brush = Brushes.Aqua;
                TSSuperTrend1.Plots[1].Brush = Brushes.Fuchsia;
                AddChartIndicator(TrendMagicTSmod1);
                AddChartIndicator(TrendMagicTSmod2);
                AddChartIndicator(TSSuperTrend1);
            }
        }

        protected override void OnBarUpdate()
        {
            if (BarsInProgress != 0)
                return;

             // Set 1
            if (Position.MarketPosition == MarketPosition.Flat)
            {
                BreakevenBool = false;
                STr_Trailing_Bool = false;
            }
            
            if (CurrentBars[0] < 1)
                return;

             // Set 2
            if ((Times[0][0].TimeOfDay >= Start_Time.TimeOfDay)
                 && (Times[0][0].TimeOfDay <= Stop_Time.TimeOfDay)
                 && (TrendMagicTSmod1.Direction[0] == 1)
                 && (TrendMagicTSmod2.Direction[0] == 1)
                 // Market Position
                 && ((Position.MarketPosition == MarketPosition.Flat)
                 || (Position.MarketPosition == MarketPosition.Short)))
            {
                EnterLong(Convert.ToInt32(Position_Size), @"L");
                BreakevenBool = false;
                STr_Trailing_Bool = false;
                Draw.VerticalLine(this, @"AAA Vertical line_1", 0, Brushes.Lime, DashStyleHelper.Solid, 2);
            }
            
             // Set 3
            if ((Times[0][0].TimeOfDay >= Start_Time.TimeOfDay)
                 && (Times[0][0].TimeOfDay <= Stop_Time.TimeOfDay)
                 && (TrendMagicTSmod1.Direction[0] == -1)
                 && (TrendMagicTSmod2.Direction[0] == -1)
                 // Market Position
                 && ((Position.MarketPosition == MarketPosition.Flat)
                 || (Position.MarketPosition == MarketPosition.Long)))
            {
                EnterShort(Convert.ToInt32(Position_Size), @"S");
                BreakevenBool = false;
                STr_Trailing_Bool = false;
                Draw.VerticalLine(this, @"AAA Vertical line_2", 0, Brushes.Red, DashStyleHelper.Solid, 2);
            }
            
             // Set 4
            if (Position.MarketPosition == MarketPosition.Long)
            {
                Initial_Stop_Long_Price = (Position.AveragePrice + (SL_Long * TickSize)) ;
                Breakeven_Trigger_Price_Store = Convert.ToInt32((Position.AveragePrice + (BE_Long * TickSize)) );
                STr_Trigger_Price_Store = (Position.AveragePrice + (Trail_Trigger_Long * TickSize)) ;
                Print(@"TAB4-InitialStopLongPrice-" + Convert.ToString(Initial_Stop_Long_Price) + @"-BreakevenTriggerPriceLong-" + Convert.ToString(Breakeven_Trigger_Price_Store) + @"-STrTriggerPriceLong-" + Convert.ToString(STr_Trigger_Price_Store));
            }
            
             // Set 5
            if (Position.MarketPosition == MarketPosition.Short)
            {
                Initial_Stop_Short_Price = (Position.AveragePrice + (SL_Short * TickSize)) ;
                Breakeven_Trigger_Price_Store = (Position.AveragePrice + (BE_Short * TickSize)) ;
                STr_Trigger_Price_Store = (Position.AveragePrice + (Trail_Trigger_Short * TickSize)) ;
                Print(@"TAB5-InitialStopShortPrice-" + Convert.ToString(Initial_Stop_Short_Price) + @"-BreakevenTriggerPriceShort-" + Convert.ToString(Breakeven_Trigger_Price_Store) + @"-STrTriggerPriceShort-" + Convert.ToString(STr_Trigger_Price_Store));
            }
            
             // Set 6
            if ((Position.MarketPosition == MarketPosition.Long)
                 && (BreakevenBool == false)
                 && (STr_Trailing_Bool == false))
            {
                ExitLongLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Long * TickSize)) , @"TPL", @"L");
                ExitLongStopMarket(Convert.ToInt32(Position_Size), Initial_Stop_Long_Price, @"SLL", @"L");
            }
            
             // Set 7
            if ((Position.MarketPosition == MarketPosition.Short)
                 && (BreakevenBool == false)
                 && (STr_Trailing_Bool == false))
            {
                ExitShortLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Short * TickSize)) , @"TPS", @"S");
                ExitShortStopMarket(Convert.ToInt32(Position_Size), Initial_Stop_Short_Price, @"SLS", @"S");
            }
            
             // Set 8
            if ((Position.MarketPosition == MarketPosition.Long)
                 && (High[0] >= Breakeven_Trigger_Price_Store)
                 && (BreakevenBool == false)
                 && (STr_Trailing_Bool == false)
                 && (High[0] < STr_Trigger_Price_Store))
            {
                BreakevenBool = true;
                STr_Trailing_Bool = false;
            }
            
             // Set 9
            if ((Position.MarketPosition == MarketPosition.Short)
                 && (Low[0] <= Breakeven_Trigger_Price_Store)
                 && (BreakevenBool == false)
                 && (STr_Trailing_Bool == false)
                 && (Low[0] > STr_Trigger_Price_Store))
            {
                BreakevenBool = true;
                STr_Trailing_Bool = false;
            }
            
             // Set 10
            if ((Position.MarketPosition == MarketPosition.Long)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == false))
            {
                ExitLongLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Long * TickSize)) , @"TPL", @"L");
                ExitLongStopMarket(Convert.ToInt32(Position_Size), (Position.AveragePrice + (BE_Offset)) , @"BEL", @"L");
            }
            
             // Set 11
            if ((Position.MarketPosition == MarketPosition.Short)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == false))
            {
                ExitShortLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Short * TickSize)) , @"TPS", @"S");
                ExitShortStopMarket(Convert.ToInt32(Position_Size), (Position.AveragePrice - (BE_Offset)) , @"BES", @"S");
            }
            
             // Set 12
            if ((Position.MarketPosition == MarketPosition.Long)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == false)
                 && (High[0] >= STr_Trigger_Price_Store))
            {
                STr_Trailing_Bool = true;
                BreakevenBool = true;
            }
            
             // Set 13
            if ((Position.MarketPosition == MarketPosition.Short)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == false)
                 && (Low[0] <= STr_Trigger_Price_Store))
            {
                STr_Trailing_Bool = true;
                BreakevenBool = true;
            }
            
             // Set 14
            if ((Position.MarketPosition == MarketPosition.Long)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == true))
            {
                ExitLongLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Long * TickSize)) , @"TPL", @"L");
                ExitLongStopMarket(Convert.ToInt32(Position_Size), TSSuperTrend1.UpTrend[1], @"TRSLL", @"L");
            }
            
             // Set 15
            if ((Position.MarketPosition == MarketPosition.Short)
                 && (BreakevenBool == true)
                 && (STr_Trailing_Bool == true))
            {
                ExitShortLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Short * TickSize)) , @"TPS", @"S");
                ExitShortStopMarket(Convert.ToInt32(Position_Size), TSSuperTrend2.DownTrend[1], @"TRSLS", @"S");
            }
            
        }​

[NinjaTrader_ChelseaB]

Hello Browniver,

It will be necessary to reduce and use prints and TraceOrders to identify what is causing changes.

Below is a link to a forum post on using prints to understand behavior.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

Currently, I am only aware of Bars.BarsSinceNewTradingDay having a bug which was corrected in the latest release.

[Browniver]

Chelsea... I have watched your video about debugging. Its not the classic You Tube video. I found it here in the forum and its from google cloud:

https://drive.google.com/file/d/1CClt0GsUFLIwQVhVyTD4JjJgEIwJKv6L/view?pli=1

Do you have any more of these very good videos? I know Ninjatraders You Tube channel but maybe you can give me some links to other vids you or NT made that I cannot find on You Tube.