Strategy Analyzer; Differences after restart

[Browniver]

Can somebody explain why the performance data in the summary tab changes when NT8 is restarted? What does NT8 fundamentally do that these data change with out changing anything within the strategy?

To clarify: I do an optimization and get results in the summary tab under performance. Then I restart NT8 and the data in the summary tab are totally different without having changed anything.
What did NT8 do?

[NinjaTrader_BrandonH]

Hello Browniver,

Thanks for your post.

What version of NinjaTrader are you using? Please provide the entire version number. This can be found under Help -> About (Example: 8.?.?.?)

If you test the SampleMACrossover strategy that comes default with NinjaTrader do you see the behavior occur?

I have tested running an optimization on the SampleMACrossover strategy in the Strategy Analyzer window and restarting NinjaTrader. I am seeing the same values appearing in the Strategy Analyzer window after NinjaTrader is restarted.

See this demonstration video: https://brandonh-ninjatrader.tinytak...N18yMjY3ODk2MQ

[Browniver]

I am using the latest version 1.2.1. It does not happen with the SampleMACrossover strategy.

Why does it happen with every other more complex strategy I am testing?
What does NT8 do when I restart?
What and why does it recalculate?
Why do I get different results when using the same setting and test data?

And maybe the most important question: What results are true? What numbers can I trust?

Thanks Brandon!

[NinjaTrader_BrandonH]

Hello Browniver,

Thanks for your notes.

Based on the information provided we would not be able to know what exactly is causing the behavior to occur.

If the behavior does not occur when testing the SampleMACrossover strategy but does occur when testing your custom strategy, the behavior is likely caused by some code in your script. You would need to take debugging steps by reducing code and adding prints to determine the exact line of code causing the behavior to occur.

If at any point you reduce/comment out a section of code in your script and the behavior stops, it is likely that last commented-out section of code causing the behavior. You could then further reduce that section of code and add debugging prints to narrow in on the offending code.

Below is a link to a forum post that demonstrates how to use prints to understand behavior.
https://ninjatrader.com/support/foru...121#post791121

[Browniver]

Thanks for your tips Brandon!

I have used prints when building the strategy. The strategy works the way intended. Dont have any problems when using live. IsInstantiatedOnEachOptimizationIteration is true.

But I would like to understand how NT8 works so I can find the problem.

Can you explain how Strategy Analyzer fundamentally works after restart?
Does it recalculate anything after restart?
Does it internally use different data or settings?
Even if there is something "wrong" in the code, why different results when using the same code?

I mean I do not change anything. Does it not use the same code, same data, same start and end time, same settings after restart?

Thanks for your patience!

[NinjaTrader_BrandonH]

Hello Browniver,

Thanks for your notes.

In the Support department at NinjaTrader we do not have any details on the internal logic of how the Strategy Analyzer functions.

If the SampleMACrossover script does not show the behavior but your custom strategy does, it will be some logic in your script that is causing the behavior to occur.

You must reduce the code of your script and further debug it by adding prints as noted in post # 4 to see what the exact line of code in your custom logic causing the behavior is.

[Browniver]

Thanks again Brandon and Ninjatrader!

I guess best is to find a developer on Fivver so we can debug together. There is so much to learn!

[donto]

Hello Browniver.
Do you use TickReplay when doing youre optimization or back test then that could be why results suddenly change.
to me it seems stupid that everytime you re-visit you resultst they are re-run on new instead of saving the results.

check this post for posible reason if using TickReplay.

https://forum.ninjatrader.com/forum/ninjatrader-8/platform-technical-support-aa/1279387-strategy-analyzer-results-corupted

[Browniver]

Thanks Donto! I saw you had a similar question/problem. No I dont use Tick Replay for optimization!

I have asked different people and groups now to help debug my strategies (or NT8?). We havent found a solution yet. In fact we found that this problem happens to many other strategies too.

I keep you updated!

[Browniver]

We have spent the whole weekend trying to find out why after restart the summary table shows different results as the overview table in the top. We found out that if TIME (Trading start time / Trading end time) is hardcoded this does not happen. Only when TIME is used as an input results change after restart. We did NOT optimize time. We only used it as a parameter but without changing or optimizing it.

Can anybody explain why this is happening?

[Browniver]

Brandon.. Any Ideas why it behaves like this?

[NinjaTrader_BrandonH]

Hello Browniver,

Thanks for your notes.

Please create a simple reduced test script with prints that demonstrate the behavior in question along with the exact steps and settings you are using to reproduce the behavior using the reduced test script.

We will test the reduced script you share on our end using the steps you provide to try and reproduce the behavior and investigate this matter further.

Note that a reduced copy refers to a copy of the script that contains the minimum amount of code needed to reproduce the issue. All other code is commented out or removed.

To create a copy of your script to modify, open a New > NinjaScript Editor, select your script, right-click in the Editor, select 'Save as', name the script, and click OK.

To export the script, go to Tools > Export > NinjaScript AddOn.

We look forward to assisting further.

[Browniver]

Hi Brandon

I have builded a new very simple strategy with strategy builder to shows the problem. Trying to export it I get this message:

"Error in compile process: Array dimensions exceeded supported range."


Can I also send a .cs file or upload the code here?

[NinjaTrader_BrandonH]

Hello Browniver,

Thanks for your notes.

Please remove any logic that is not necessary for reproducing the issue and test again to confirm the behavior can still be reproduced.

If the issue can still be reproduced, please share the open source .CS file so that I may investigate this matter further. Custom NinjaScript indicator files could be found in the Documents/NInjaTrader 8/bin/Custom/Strategies folder. Also, send the exact steps and settings you are using to reproduce the behavior in question.

Note that a reduced copy refers to a copy of the script that contains the minimum amount of code needed to reproduce the issue. All other code is commented out or removed.

To create a copy of your script to modify, open a New > NinjaScript Editor, select your script, right-click in the Editor, select 'Save as', name the script, and click OK.

[Browniver]

Thanks Brandon!

Attached you will find a very simple strategy built with strategy Builder using a NT8 standard indicator. After optimizing the results in the overview panel on the top match the summary/performance data in the middle of the strategy analyzer. After restart however data do systematically NOT match anymore. Can you reproduce this behavior?


I think it has to to with Start_Time and Stop_Time not being hardcoded.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.Indicators;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Strategies in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Strategies
{
    public class NT8Test : Strategy
    {
        private double Initial_Stop_Long_Price;
        private double Initial_Stop_Short_Price;
        private double Breakeven_Trigger_Price_Store;
        private bool BreakevenBool;

        private RSI RSI1;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Enter the description for your new custom Strategy here.";
                Name                                        = "NT8Test";
                Calculate                                    = Calculate.OnPriceChange;
                EntriesPerDirection                            = 9;
                EntryHandling                                = EntryHandling.AllEntries;
                IsExitOnSessionCloseStrategy                = true;
                ExitOnSessionCloseSeconds                    = 120;
                IsFillLimitOnTouch                            = false;
                MaximumBarsLookBack                            = MaximumBarsLookBack.TwoHundredFiftySix;
                OrderFillResolution                            = OrderFillResolution.Standard;
                Slippage                                    = 0;
                StartBehavior                                = StartBehavior.WaitUntilFlatSynchronizeAccount;
                TimeInForce                                    = TimeInForce.Gtc;
                TraceOrders                                    = true;
                RealtimeErrorHandling                        = RealtimeErrorHandling.StopCancelClose;
                StopTargetHandling                            = StopTargetHandling.PerEntryExecution;
                BarsRequiredToTrade                            = 20;
                // Disable this property for performance gains in Strategy Analyzer optimizations
                // See the Help Guide for additional information
                IsInstantiatedOnEachOptimizationIteration    = true;
                Position_Size                    = 1;
                Start_Time                        = DateTime.Parse("00:00", System.Globalization.CultureInfo.InvariantCulture);
                Stop_Time                        = DateTime.Parse("23:00", System.Globalization.CultureInfo.InvariantCulture);
                SL_Long                    = -50;
                SL_Short                    = 50;
                TP_Long                    = 50;
                TP_Short                    = -50;
                Initial_Stop_Long_Price                    = 0;
                Initial_Stop_Short_Price                    = 0;
                
            }
            else if (State == State.Configure)
            {
            }
            else if (State == State.DataLoaded)
            {                
                RSI1                = RSI(Close, 14, 3);
                RSI1.Plots[0].Brush = Brushes.DodgerBlue;
                RSI1.Plots[1].Brush = Brushes.Goldenrod;
                AddChartIndicator(RSI1);
            }
        }

        protected override void OnBarUpdate()
        {
            if (BarsInProgress != 0)
                return;

            if (CurrentBars[0] < 1)
                return;

             // Set 1
            if ((Position.MarketPosition == MarketPosition.Flat)
                 && (Times[0][0].TimeOfDay >= Start_Time.TimeOfDay)
                 && (Times[0][0].TimeOfDay <= Stop_Time.TimeOfDay)
                 && (CrossAbove(RSI1.Default, 20, 1)))
            {
                EnterLong(Convert.ToInt32(Position_Size), @"L");
                
            }
            
             // Set 2
            if ((Position.MarketPosition == MarketPosition.Flat)
                 && (Times[0][0].TimeOfDay >= Start_Time.TimeOfDay)
                 && (Times[0][0].TimeOfDay <= Stop_Time.TimeOfDay)
                 && (CrossBelow(RSI1.Default, 80, 1)))
            {
                EnterShort(Convert.ToInt32(Position_Size), @"S");
                
            }
            
             // Set 3
            if (Position.MarketPosition == MarketPosition.Long)
            {
                Initial_Stop_Long_Price = (Position.AveragePrice + (SL_Long * TickSize)) ;
                Print(@"TAB4-" + Convert.ToString(new DateTime(2023, 12, 31)) + "-" + Convert.ToString(new TimeSpan(19, 13, 0)) + @"-InitialStopLong-" + Convert.ToString(Initial_Stop_Long_Price));
            }
            
             // Set 4
            if (Position.MarketPosition == MarketPosition.Short)
            {
                Initial_Stop_Short_Price = (Position.AveragePrice + (SL_Short * TickSize)) ;
                Print(@"TAB5-" + Convert.ToString(new DateTime(2023, 12, 31)) + "-" + Convert.ToString(new TimeSpan(19, 7, 0)) + @"-InitialStopShort-" + Convert.ToString(Initial_Stop_Short_Price));
            }
            
             // Set 5
            if (Position.MarketPosition == MarketPosition.Long)
            {
                ExitLongLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Long * TickSize)) , @"TPL", @"L");
                ExitLongStopMarket(Convert.ToInt32(Position_Size), Initial_Stop_Long_Price, @"SLL", @"L");
            }
            
             // Set 6
            if (Position.MarketPosition == MarketPosition.Short)
            {
                ExitShortLimit(Convert.ToInt32(Position_Size), (Position.AveragePrice + (TP_Short * TickSize)) , @"TPS", @"S");
                ExitShortStopMarket(Convert.ToInt32(Position_Size), Initial_Stop_Short_Price, @"SLS", @"S");
            }
            
        }

        #region Properties
        [NinjaScriptProperty]
        [Range(1, int.MaxValue)]
        [Display(Name="Position_Size", Order=1, GroupName="Parameters")]
        public int Position_Size
        { get; set; }

        [NinjaScriptProperty]
        [PropertyEditor("NinjaTrader.Gui.Tools.TimeEditorKey")]
        [Display(Name="Start_Time", Order=2, GroupName="Parameters")]
        public DateTime Start_Time
        { get; set; }

        [NinjaScriptProperty]
        [PropertyEditor("NinjaTrader.Gui.Tools.TimeEditorKey")]
        [Display(Name="Stop_Time", Order=3, GroupName="Parameters")]
        public DateTime Stop_Time
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="SL_Long", Order=4, GroupName="Parameters")]
        public int SL_Long
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="SL_Short", Order=5, GroupName="Parameters")]
        public int SL_Short
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="TP_Long", Order=6, GroupName="Parameters")]
        public int TP_Long
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="TP_Short", Order=7, GroupName="Parameters")]
        public int TP_Short
        { get; set; }

        
        #endregion

    }
}
​