Announcement

Collapse

Looking for a User App or Add-On built by the NinjaTrader community?

Visit NinjaTrader EcoSystem and our free User App Share!

Have a question for the NinjaScript developer community? Open a new thread in our NinjaScript File Sharing Discussion Forum!
See more
See less

Partner 728x90

Collapse

TickQ & TrinQ values different when accessed from BarUpdate and from Historical Data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    TickQ & TrinQ values different when accessed from BarUpdate and from Historical Data

    Happy New Year everyone.

    My first post on here and hopefully someone can help me out.

    I'm trying to do some custom optimization by looping back through historical data and I've found that the values returned for TickQ and TrinQ are different based on whether they are called as OnBarUpdate is stepping through the historical data the first time, or when they are called at the end of the historical data updates.

    You can see this from the output for exactly the same bars of DDD:

    First you can see the data output when called on Bar[0] as it is being stepped through the first time as the strategy is enabled:

    DateofBar 20120925 TimeofBar 0 is 140000 Close = 36.24 Open = 36.11
    MACdiff of bar 0 is -0.00175320328820028 MACDdiff of bar 0 is -0.00175320328820028
    MACD of bar 0 is 0.336213598844452 MACDAvg of bar 0 is 0.337966802132652
    ADX of bar 0 is 27.1365507209919
    Tick 264
    Trin 1.58


    Then when I find the exactly same bar when OnBarUpdate is pointing at the end of the historical data by looking back at bar 835, you can see the timestamp, MACD, and ADX values are exactly the same, but for some reason the Tick and Trin values are now different:

    DateofBar 20120925 TimeofBar 835 is 140000 Close = 36.24 Open = 36.11
    MACdiff of bar 835 is -0.00175320328820028 MACDdiff of bar 835 is -0.00175320328820028
    MACD of bar 835 is 0.336213598844452 MACDAvg of bar 835 is 0.337966802132652
    ADX of bar 835 is 27.1365507209919
    Tick -119
    Trin 1.35


    So I checked the data in the Historical Data Manager to see which values are correct, and you can see the first step through values are correct:
    Real Tick from Historical Data Manager = 264
    Real Trin from Historical Data Manager = 1.58

    I have no idea what's going. Tick and Trin values are wrong for all bars when looked at from the end of historical data. I'm using exactly the same code for each output. I can think of a work around by storing the real TickQ and TrinQ values in an array as OnBarUpdate is stepping through the data the first time, but I really want to understand what's going on here.

    Here's the relevant code:

    protected override void Initialize()
    {
    CalculateOnBarClose = true;
    EntryHandling = EntryHandling.UniqueEntries;
    Add ("^TICKQ", PeriodType.Minute, 30);
    Add ("^TRINQ", PeriodType.Minute, 30);
    Add (PeriodType.Minute, 5);
    }

    protected override void OnBarUpdate()
    {
    if (CurrentBars[0] <= BarsRequired ||
    CurrentBars[1] <= BarsRequired || CurrentBars[2]
    <= BarsRequired || CurrentBars[3]
    <= BarsRequired)
    return;

    if (BarsInProgress == 0) // Print values as OnBarUpdate steps through on startup
    {
    Print("DateofBar "+ ToDay(Time[0]) + " TimeofBar " + 0 + " is " + ToTime(Time[0]) + " Close = " + Close[0] + " Open = " + Open[0]);
    Print("MACdiff of bar " + 0 + " is " + MACD(12,26,9).Diff[0] + " MACDdiff of bar " + (0) + " is " + MACD(12,26,9).Diff[0]);
    Print("MACD of bar " + 0 + " is " + MACD(12,26,9)[0] + " MACDAvg of bar " + (0) + " is " + MACD(12,26,9).Avg[0]);
    Print("ADX of bar " + 0 + " is " + ADX(adxvalue)[0]);
    Print("Tick " + (Closes[1][0]));
    Print("Trin " + (Closes[2][0]));
    Print("");
    }

    if (Historical == true
    && (Count - 2 == CurrentBar)) // Print values when OnBarUpdate has reached the end of historical data
    {
    if (BarsInProgress == 0)
    {
    int loopsbar2 = Count - 26;
    for(int i = loopsbar2; i >= 0; i--)
    {
    Print("DateofBar "+ ToDay(Time[i]) + " TimeofBar " + i + " is " + ToTime(Time[i]) + " Close = " + Close[i] + " Open = " + Open[i]);
    Print("MACdiff of bar " + i + " is " + MACD(12,26,9).Diff[i] + " MACDdiff of bar " + (i) + " is " + MACD(12,26,9).Diff[i]);
    Print("MACD of bar " + i + " is " + MACD(12,26,9)[i] + " MACDAvg of bar " + (i) + " is " + MACD(12,26,9).Avg[i]);
    Print("ADX of bar " + i + " is " + ADX(adxvalue)[i]);
    Print("Tick " + (Closes[1][i]));
    Print("Trin " + (Closes[2][i]));
    Print("");
    }
    }
    }
    }

    #2
    Hello travel767,
    Welcome to the forum and I am happy to assists you.

    I am getting the below error when I try applying the strategy.

    **NT** Error on calling 'OnBarUpdate' method for strategy 'Test/6bc49e9c243d46a2a82952a87c1e8282': You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
    Can you please upload a working copy of the code depicting the exact scenario.

    Also please upload the actual strategy only.
    JoydeepNinjaTrader Customer Service

    Comment


      #3
      Thanks for the quick reply.

      Here's a working version of the code in the attached file. In the output screen, look at any two bars that have the same date and time stamp and you can see that the trick and trin values are different.
      Attached Files

      Comment


        #4
        Hello travel767,
        Thanks for the code. However when I apply it on the chart, I am getting the following error.

        **NT** Error on calling 'OnBarUpdate' method for strategy 'TickTrinTesting/c7ff74156cc14f9baa7b3e1b1223521f': You are accessing an index with a value that is invalid since its out of range. I.E. accessing a series [barsAgo] with a value of 5 when there are only 4 bars on the chart.
        Can you please let me know the exact settings (parameters) of the strategy you are using so that I can replicate at my end.

        Also please let me know what is the primary instrument, and how many days of data you are loading to test the indicator.
        JoydeepNinjaTrader Customer Service

        Comment


          #5
          I don't know why the uploaded code gives you an error. It runs fine for me when run from a chart and the strategy panel from the control center. I can't see anything that would out of range.

          I'm using 30min bars, and 100 days of data. However, I've been able to replicate the difference in values using different amounts of historical data and ARMH, but for some reason it works fine for SWI. I will try more instruments...

          Comment


            #6
            Hello travel767,
            Thanks for the information.

            I can replicate it at my end and will get back to you with my findings.

            Thanks for your patience.
            JoydeepNinjaTrader Customer Service

            Comment


              #7
              Hello travel767,
              If I modify the OnBarUpdate secotion of the code as below then I am getting the following prints.
              DateofBar 20120925 TimeofBar 0 is 140000 Close = 750.2 Open = 751.29
              MACdiff of bar 0 is -1.38280904791404 MACDdiff of bar 0 is -0.95105226382219
              MACD of bar 0 is 2.68593438584423 MACDAvg of bar 0 is 4.06874343375828
              ADX of bar 0 is 56.8137360121342
              Tick -864 Time 9/25/12 2:00:00 PM
              Trin 2.97 Time 9/25/12 2:00:00 PM

              DateofBar 20120925 TimeofBar 569 is 140000 Close = 750.2 Open = 751.29
              MACdiff of bar 569 is -1.38280904791404 MACDdiff of bar 569 is -0.95105226382219
              MACD of bar 569 is 2.68593438584423 MACDAvg of bar 569 is 4.06874343375828
              ADX of bar 569 is 56.8137360121342
              Tick -30 Time 10/24/12 10:00:00 AM
              Trin 1.11 Time 10/24/12 10:00:00 AM

              Code:
              protected override void OnBarUpdate()
              {
              	
              	
              	
              	
              	if (Historical == true
              		&& BarsInProgress == 0)				 // Print values as OnBarUpdate steps through on startup
              	{		
              	Print("DateofBar "+ ToDay(Time[0]) + " TimeofBar " + 0 + " is " + ToTime(Time[0]) + " Close = " + Close[0] + " Open = " + Open[0]);
              	Print("MACdiff of bar " + 0 + " is " + MACD(12,26,9).Diff[0] + " MACDdiff of bar " + (0) + " is " + MACD(12,26,9).Diff[1]);
              	Print("MACD of bar " + 0 + " is " + MACD(12,26,9)[0] + " MACDAvg of bar " + (0) + " is " + MACD(12,26,9).Avg[0]);	
              	Print("ADX of bar " + 0 + " is " + ADX(adxvalue)[0]);
              	[B]Print("Tick " + (Closes[1][0]).ToString() + " Time  " + (Times[1][0]).ToString());
              	Print("Trin " + (Closes[2][0]).ToString() + " Time " + (Times[2][0]).ToString());[/B]
              	Print("");	
              	}		
              	
              	if (Historical == true
              		&& (Count - 2 == CurrentBar))		 // Print values when OnBarUpdate has reached the end of historical data																															
              	{
              		if (BarsInProgress == 0)
              		{						
              				int loopsbar2 = Count - 26;																		
              					for(int i = loopsbar2; i >= 0; i--)
              					{
              					Print("DateofBar "+ ToDay(Time[i]) + " TimeofBar " + i + " is " + ToTime(Time[i]) + " Close = " + Close[i] + " Open = " + Open[i]);
              					Print("MACdiff of bar " + i + " is " + MACD(12,26,9).Diff[i] + " MACDdiff of bar " + (i) + " is " + MACD(12,26,9).Diff[i+1]);
              					Print("MACD of bar " + i + " is " + MACD(12,26,9)[i] + " MACDAvg of bar " + (i) + " is " + MACD(12,26,9).Avg[i]);	
              					Print("ADX of bar " + i + " is " + ADX(adxvalue)[i]);
              					[B]Print("Tick " + (Closes[1][i]).ToString() + " Time  " + (Times[1][i]).ToString());
              					Print("Trin " + (Closes[2][i]).ToString() + " Time  " + (Times[2][i]).ToString());[/B]
              					Print("");	
              					}
              		}
              	}
              	
              	
              	
              }
              The loop is referencing to the wrong bars and thus the anomaly.
              JoydeepNinjaTrader Customer Service

              Comment


                #8
                Thanks!
                .......................

                Comment


                  #9
                  After looking closer at your modified code it just confirms the error that I'm reporting.

                  In the loop, your modified code is still referencing the same index number of the bars [i] for Tick and Trin, which should be exactly the same as the primary data series like it was in the above code.

                  But for some reason the time of Tick and Trin has now changed....

                  Why are the time and date stamps the same when OnBarUpdate is looping through index [0], but then different when accessed from index [i].
                  Last edited by travel767; 12-31-2012, 10:21 AM.

                  Comment


                    #10
                    Hello travel767,
                    It could be due to missing bars, etc.
                    If you reload the historical data using the download feature of NinjaTrader and then what values you are getting.
                    JoydeepNinjaTrader Customer Service

                    Comment


                      #11
                      Joydeep,

                      I have found the problem using your code modifications to print the time above and look back to see when the data becomes out-of-sync. You're right, it is definitely because of missing bars with TickQ and TrinQ, but the missing bars are caused from the way NinjaTrader (or IB) handled the 1300 close on Dec. 24th. The data is sync'd from the end of Dec 31st to the beginning of Dec 26th.

                      When the markets closed early on Dec 24th, TickQ and TrinQ stopped printing because they have no afterhours values.

                      But the aftermarket for equites continued after 1300, and because the session template for US Equities RTH goes until 1600, those values were included in the regular session. I'm not sure if those were real trades or just the way NinjaTrader (or my IB data feed) handled 1300-1600 Dec 24th, but from the Historical Data Manager all equites have at least 100 share trades every minute in that time period, where TickQ and TrinQ stopped printing.

                      So the bottom line is counting the afterhours equity trading 1300-1400 Dec24th in the RTH session creates an out-of-sync condition between the equity bar numbers and the index bar numbers which stopped at 1300.

                      So to fix this problem, either the equities printing needs to stop, or the index values need to keep printing during early closings. I think the best fix would be for the session templates to include provisions for early closing holidays to filter out those afternoon trades that are classed as afterhours, so the equities and index bar numbers can stay in-sync.

                      What do you think?

                      Comment


                        #12
                        I just realized that I get the correct TickQ and TrinQ values if I store them in an array as OnBarUpdate is stepping through the data.

                        So there's also a difference in the way NinjaTrader handles missing bars depending on if the data is being called as OnBarUpdate is stepping through the data the first time going forwards, or if it's being called as previous bars going backwards.

                        For example, here's the data as OnBarUpdate is stepping through the bars the first time on Bar[0]. There is no bar data for TickQ and TrinQ after 1300 on Dec 24th, but you can see OnBarUpdate still uses the last bar's values.

                        DateofBar 20121224 TimeofBar 0 is 130000 Close = 51.89 Open = 52.08
                        MACdiff of bar 0 is 0.0898452943154597 MACDdiff of bar 1 is 0.137298540689719
                        MACD of bar 0 is 0.309939540195629 MACDAvg of bar 0 is 0.220094245880169
                        ADX of bar 0 is 23.1999612789376
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 133000 Close = 51.9 Open = 51.89 (Afterhours now, there shouldn't be any bar[0]'s until Dec 26th)
                        MACdiff of bar 0 is 0.0540242994306767 MACDdiff of bar 1 is 0.0898452943154597
                        MACD of bar 0 is 0.287624620168515 MACDAvg of bar 0 is 0.233600320737838
                        ADX of bar 0 is 23.4476485009558
                        Tick 320 Time 12/24/2012 1:00:00 PM (Actual bars of TickQ and TrinQ missing from here on, but last data is still called)
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 140000 Close = 51.9 Open = 51.9
                        MACdiff of bar 0 is 0.0266106612240186 MACDdiff of bar 1 is 0.0540242994306767
                        MACD of bar 0 is 0.266863647267861 MACDAvg of bar 0 is 0.240252986043843
                        ADX of bar 0 is 23.6788232415062
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 143000 Close = 51.9 Open = 51.9
                        MACdiff of bar 0 is 0.0058430084396201 MACDdiff of bar 1 is 0.0266106612240186
                        MACD of bar 0 is 0.247556746593368 MACDAvg of bar 0 is 0.241713738153748
                        ADX of bar 0 is 23.8945863326865
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 150000 Close = 51.9 Open = 51.9
                        MACdiff of bar 0 is -0.00968371215852748 MACDdiff of bar 1 is 0.0058430084396201
                        MACD of bar 0 is 0.229609097955588 MACDAvg of bar 0 is 0.239292810114116
                        ADX of bar 0 is 24.0959652177882
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 153000 Close = 51.9 Open = 51.9
                        MACdiff of bar 0 is -0.0210895105067898 MACDdiff of bar 1 is -0.00968371215852748
                        MACD of bar 0 is 0.212930921980629 MACDAvg of bar 0 is 0.234020432487419
                        ADX of bar 0 is 24.2839188438831
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121224 TimeofBar 0 is 160000 Close = 51.9 Open = 51.9
                        MACdiff of bar 0 is -0.029266416567338 MACDdiff of bar 1 is -0.0210895105067898
                        MACD of bar 0 is 0.197437411778246 MACDAvg of bar 0 is 0.226703828345584
                        ADX of bar 0 is 24.4593422282383
                        Tick 320 Time 12/24/2012 1:00:00 PM
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121226 TimeofBar 0 is 100000 Close = 52.81 Open = 51.9
                        MACdiff of bar 0 is 0.0231499095888187 MACDdiff of bar 1 is -0.029266416567338
                        MACD of bar 0 is 0.255641215331607 MACDAvg of bar 0 is 0.232491305742789
                        ADX of bar 0 is 25.7991368357584
                        Tick 100 Time 12/26/2012 10:00:00 AM
                        Trin 0.48 Time 12/26/2012 10:00:00 AM

                        DateofBar 20121226 TimeofBar 0 is 103000 Close = 52.47 Open = 52.8
                        MACdiff of bar 0 is 0.0309723062826792 MACDdiff of bar 1 is 0.0231499095888187
                        MACD of bar 0 is 0.271206688596138 MACDAvg of bar 0 is 0.240234382313459
                        ADX of bar 0 is 27.1427937840349
                        Tick 36 Time 12/26/2012 10:30:00 AM
                        Trin 0.77 Time 12/26/2012 10:30:00 AM

                        But here's the data when called going backwards after OnBarUpdate has reached the end, and you can see it gets out of sync at 1600 Dec 24 (when read backwards):

                        DateofBar 20121224 TimeofBar 52 is 153000 Close = 51.9 Open = 51.9
                        MACdiff of bar 52 is -0.0210895105067898 MACDdiff of bar 52 is -0.00968371215852748
                        MACD of bar 52 is 0.212930921980629 MACDAvg of bar 52 is 0.234020432487419
                        ADX of bar 52 is 24.2839188438831
                        Tick 24 Time 12/24/2012 12:30:00 PM (1530 bar missing, but 1230 bar is called instead of being populated with 1300 bar like it was when going forwards, so bars are now out-of-sync)
                        Trin 0.99 Time 12/24/2012 12:30:00 PM

                        DateofBar 20121224 TimeofBar 51 is 160000 Close = 51.9 Open = 51.9
                        MACdiff of bar 51 is -0.029266416567338 MACDdiff of bar 51 is -0.0210895105067898
                        MACD of bar 51 is 0.197437411778246 MACDAvg of bar 51 is 0.226703828345584
                        ADX of bar 51 is 24.4593422282383
                        Tick 320 Time 12/24/2012 1:00:00 PM (1600 bar missing, so 1300 bar is called, this data is correct)
                        Trin 0.98 Time 12/24/2012 1:00:00 PM

                        DateofBar 20121226 TimeofBar 50 is 100000 Close = 52.81 Open = 51.9
                        MACdiff of bar 50 is 0.0231499095888187 MACDdiff of bar 50 is -0.029266416567338
                        MACD of bar 50 is 0.255641215331607 MACDAvg of bar 50 is 0.232491305742789
                        ADX of bar 50 is 25.7991368357584
                        Tick 100 Time 12/26/2012 10:00:00 AM
                        Trin 0.48 Time 12/26/2012 10:00:00 AM

                        DateofBar 20121226 TimeofBar 49 is 103000 Close = 52.47 Open = 52.8
                        MACdiff of bar 49 is 0.0309723062826792 MACDdiff of bar 49 is 0.0231499095888187
                        MACD of bar 49 is 0.271206688596138 MACDAvg of bar 49 is 0.240234382313459
                        ADX of bar 49 is 27.1427937840349
                        Tick 36 Time 12/26/2012 10:30:00 AM
                        Trin 0.77 Time 12/26/2012 10:30:00 AM

                        So there's also an issues with how Ninjatrader handles missing bars - When going forwards the last bar's data is called as the current bar's data, but when going backwards, the missing bars are not populated with the last recorded data...

                        I hope this makes sense

                        Comment


                          #13
                          Hello travel767,
                          Unfortunately it is expected in case of truncated sessions.

                          As an alternative if you try using the GetBar method to retrive the past bars, then are you able to get the correct values.

                          Please refer to our help guide to know more about the GetBar method.
                          JoydeepNinjaTrader Customer Service

                          Comment


                            #14
                            Got it. Thanks again!

                            Comment

                            Latest Posts

                            Collapse

                            Topics Statistics Last Post
                            Started by knighty6508, 05-10-2024, 01:20 AM
                            4 responses
                            24 views
                            0 likes
                            Last Post knighty6508  
                            Started by OllieFeraher, 05-09-2024, 11:14 AM
                            6 responses
                            19 views
                            0 likes
                            Last Post OllieFeraher  
                            Started by PaulMohn, 05-02-2024, 06:59 PM
                            2 responses
                            43 views
                            0 likes
                            Last Post PaulMohn  
                            Started by ETFVoyageur, Today, 02:10 AM
                            0 responses
                            11 views
                            0 likes
                            Last Post ETFVoyageur  
                            Started by rayyyu12, Today, 12:47 AM
                            0 responses
                            8 views
                            0 likes
                            Last Post rayyyu12  
                            Working...
                            X