C# Newbie - Please Help Me Understand Using Bar Data

[bltdavid]

So if I'm accessing the High at Count - 1, and it errors out -- what am I doing wrong here?

Using 'Count-1' is wrong, use 'CurrentBar-1' instead.

Why?
When you reach State.DataLoaded, NT has loaded all historical bars,
but you don't have access to these historical bars just yet. The value in
Count represents the number of historical bars waiting in the wings.

Waiting for what?
For OnBarUpdate, silly. Your code gets a chance to 'see' each
new bar added to the chart via OnBarUpdate. You are not allowed
to peek ahead. Just because you know the value Count doesn't
really mean much -- use CurrentBar instead, this property is
incremented by 1 each time a bar is closed -- and what happens
when a bar closes? Well, doh, NT calls the OnBarUpdate method.

Make sense?

[bltdavid]

Assuming Calculate is set to Calculate.OnBarClose ...

On the very first call to OnBarUpdate, the very first one, CurrentBar is always 0.
So, what has happened? A bar has closed, OnBarUpdate is called, and this very
first bar starts out with CurrentBar being 0.

When the 2nd bar closes, OnBarUpdate is called, and CurrentBar will be 1.
When the 3rd bar closes, OnBarUpdate is called, and CurrentBar will be 2.
etc etc

What else do we know about these first few bars?
They are historical (Ie, as long as State is State.Historical, then they are historical)
The very first bar has no previous bar.

What about 'Count'?
While you're in State.Historical, the value of Count doesn't change.

When you start State.Realtime, Count will start incrementing by 1
each time a bar closes.

Good reading here and here.

[TallTim]

I thank you both for your patience.

I will look at the links provided.

I guess when I saw things in the code like "if (State == State.DataLoaded)" I thought it meant "okay, all bars loaded, we're ready to access!" But that assumption wasn't correct for the reasons you provided.

I'll try to bang my head against this a bit more...

Update -- so, I did get my "call the oldest bar after loading" to work.

My code in OnBarUpdate():

if (CurrentBar < (Count-2))
{
    Value[0] = 0; // Indicator value zero if loading bars
    return;
}
else
{
    Print("Close for way back bar :" + (Count - 2).ToString() + " is: " + Close[Count-2]);​
}

This is on a 30 minute bar loading 5 days which totaled a Count of 521 bars. If I understand correctly, the reason 521 - 2 = 519 works is because first, we are using zero indexes (so subtract one) and second, as the chart "fills" in there is technically the bar "-1" which is being 'built' by the tick data that is processed into my timeframe.

So for my fellow newbies the array as it processes historical ticks is:

etc..,[2],[1]<--- Any already processed bars index increment on timeline this way, [ 0 = "Lead" processed bar ], [ -1 = Incoming tick data, not 'built' yet ]

Feel free to correct me if this is wrong. But it helps me understand how this works.

[TallTim]

I just had my breakthrough moment.

Just from absorbing things and finally getting it in my head straight.

Assuming that you have OnBarUpdate() using the Calculate.OnBarClose setting, the following is true:

Oldest bar displayed on the chart == CurrentBar -- I would personally like to take whoever named this and shake them by their shoulders until their teeth fell out, but I get it now.

Newest updated bar on the chart after it closes == Index 0

To get the proper starting point for any custom indicator calculation, you take your Interval in bars and subtract one -- this becomes the offset index you need to start at to begin calculations.

Feel like I climbed a mountain, but I'd like to thank everyone who contributed. Now to explore even more errors from mistakes in C# < laugh >.

[trdninstyle]

Wow, I learned a lot just reading this entire post, and I actually tracked it. I must be getting there.

There's a saying I can't quite remember but it goes something like this:

An inexperienced man can't hope to stand with an experienced man with an argument. Another words, all of You's are above my head.
Thanks for the education, guys.

@Bill

[TallTim]

Another addendum about certain properties relating to Bar indexes -- just laid it out differently so it makes more sense to me, maybe this will help someone:

Consider a chart that has six bars, five have already closed and one is "building".

Bar Count:

[1] [2] [3] [4] [5] [6 - In progress]

"Current Bar" property: (I swear, I still want to yell at the person who named this.)

[0] [1] [2] [3] [4] [ -1 - In progress, when closed, it becomes CurrentBar Index 5 ]

This is why Count - 2 is the oldest bar on the chart, in terms of indexing, Subtract one for starting at zero, and one more to account for "building" bar.

You'll also see "Bars Ago" as a required index value when using things like Close[<index num>] and such:

BarsAgo returns the newest closed bar with index 0:

[4] [3] [2] [1] [0] [ -1 - In progress, when closed it becomes Bars Ago Index 0]

---------------------------------------

I think 90% of the problems posted on this forum come from not getting the structure of these indexes, and I don't blame them -- this is really a frustrating way to deal with bar data.

[TallTim]

EDIT -- DO NOT USE ADD DATA SERIES if you don't need it!! This was causing OnBarUpdate() to execute more than once, which may trip you up if you're not expecting it. I am a total n00b and was using it, even though there wasn't a good reason to. Thanks, bltdavid!!

Also one more thing -- is there a reason that OnBarUpdate() when using Calculate.OnBarClose setting runs things twice?

I've been fighting this the whole damn time, and its supremely annoying.

Seriously, just put a simple Print("This should only appear once"); in an otherwise empty OnBarUpdate and watch the output log print it twice.

WHY?

Edit: ( I know why, see below... doh!)

[bltdavid]

Can you attach the exact script you are using?

[TallTim]

Can you attach the exact script you are using?

I think I know what is going on -- maybe.

A separate test reveals that it is executing as it loads the bar data on the chart -- so I might have a circumstance somewhere that as it is loading I'm seeing repeats because my logic isn't containing the processing phase properly.

For instance -- here's a fresh indicator "Bars With Currant Glaze" that has a Print statement at the beginning:

namespace NinjaTrader.NinjaScript.Indicators
{
    public class BarsWithCurrantGlaze : Indicator
    {
        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Please tell whoever made Current Bar a thing to pack up their things and take their coat on the way out.";
                Name                                        = "BarsWithCurrantGlaze";
                Calculate                                    = Calculate.OnBarClose;
                IsOverlay                                    = false;
                DisplayInDataBox                            = true;
                DrawOnPricePanel                            = true;
                DrawHorizontalGridLines                        = true;
                DrawVerticalGridLines                        = true;
                PaintPriceMarkers                            = true;
                ScaleJustification                            = NinjaTrader.Gui.Chart.ScaleJustification.Right;
                //Disable this property if your indicator requires custom values that cumulate with each new market data event.
                //See Help Guide for additional information.
                IsSuspendedWhileInactive                    = true;
                NumberOfCurrantsOnYourBars                    = 15;
                AddPlot(Brushes.Orange, "CurrantSweetness");
            }
            else if (State == State.Configure)
            {
            }
        }

        protected override void OnBarUpdate()
        {
            //Add your custom indicator logic here.
            Print("Bars with currant glaze is so delicious!");

            if (CurrentBar < (Count - 2)) // Waiting until whole chart is loaded...
            {
                Value[0] = 0; // Indicator value zero if loading bars
                return;
            }
            else            
            {

            }
        }

        #region Properties
        [NinjaScriptProperty]
        [Range(2, int.MaxValue)]
        [Display(Name="NumberOfCurrantsOnYourBars", Description="Delicious Currants Everywhere", Order=1, GroupName="Parameters")]
        public int NumberOfCurrantsOnYourBars
        { get; set; }

        [Browsable(false)]
        [XmlIgnore]
        public Series<double> CurrantSweetness
        {
            get { return Values[0]; }
        }
        #endregion

    }
}​

This results in an output log of:


This is the chart I've loaded the indicator on -- data is paused so it isn't "fresh".

As you can see, there are nine loaded bars, and one "in progress" bar.

When I move the Print statement into the logic that waits for full load, then I get an output log of:



So yeah, I have some debugging to do -- just was going nuts for a short while. Thanks for the reply though - it forced me to do a "clean" test to make sure I could replicate.

[TallTim]

I don't want to start a new thread -- since it is relevant to using Bar Data -- what is the simplest way to implement a "sliding window" of a given interval that doesn't change during iteration over a set of bar data?

For instance, if I had a total of 13 bars loaded and my 'window' was an interval of say, 5 - how would I drive a loop to process that -- while doing calculations using the interval every iteration?

I want to start at the highest index (since that is the oldest historical bar) and decrement by 1 until the interval end point is 0.

It seems more complex than I initially thought, so any suggestions are welcome. Just feel like there has to be an easier way.

--------------

Edit -- Upon further thinking I'd like to harness how OnBarUpdate() loads bar data, but I still don't know how to "pause" it as I calculate my intervals. Right now, its a firehose that spews out all the data and then I'm left with the problem stated above. Maybe a veteran can guide me to the right answer, its probably really obvious but I'm missing it right now...

-------------

Further Edit -- I think what I'm describing in C# is "Array Slicing" but I'll have to investigate....

[bltdavid]

No array slicing involved, just a simple loop.

(EDIT: Be careful, they may look the same,
but a 'Series' is absolutely not an 'Array')

private int Interval = 5;

protected override void OnBarUpdate()
{
    // guard code
    if (CurrentBar < Interval)
        return;

    // (A) process oldest closed bar first
    for (int j = Interval - 1; j >= 0; --j)
        Print(string.Format(" BarsAgo={0}: Close={1}", j, Close[j]));

    // (B) process most recent closed bar first
    for (int j = 0; j < Interval; ++j)
        Print(string.Format(" BarsAgo={0}: Close={1}", j, Close[j]));
​}

The code in (A) is what you requested, but processing your sliding
window is usually not done in that direction.

Why?
Good question -- it probably doesn't matter to the algorithm which
direction you process the sliding window, so using the code in (B)
is more commonly used, probably because it just looks cleaner, or,
as programmers might say, (B) is an idiom.

[bltdavid]

Try studying @CCI.cs -- that code has a loop that starts at the oldest bar in the interval.

[bltdavid]

See also loop inside @StdDev.cs -- very similar to looping in @CCI.cs.

As far as being common (aka an 'idiom'), perhaps I misspoke.



Oh well ... my bad.

[TallTim]

Appreciate the replies bitdavid -- its a bit late for me here but I actually figured it out I think... I'll update for the educational value after I sleep a bit ZZzzzz...

(I've been programming in a language that doesn't use zero indexes, so its been a bit of a mind-warp getting re-accustomed to that.)

[TallTim]

EDIT -- DO NOT USE ADD DATA SERIES if you don't need it!! This was causing OnBarUpdate() to execute more than once, which may trip you up if you're not expecting it. I am a total n00b and was using it, even though there wasn't a good reason to. Thanks, bltdavid!!

As promised -- here is how I made a "sliding window" based on a user-input "Interval" in bars.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Indicators in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Indicators
{    
    public class TTBullBear : Indicator
    {
        // Variable declarations
        int startIndex = 0;
        int endIndex = 0;
        int myCount = 0;

//-------------------------------------------- Main Callbacks -----------------------------------------------------
        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Testing Indicator Stuff";
                Name                                        = "TTBullBear";
                Calculate                                    = Calculate.OnBarClose;
                IsOverlay                                    = false;
                DisplayInDataBox                            = true;
                DrawOnPricePanel                            = true;
                DrawHorizontalGridLines                        = true;
                DrawVerticalGridLines                        = true;
                PaintPriceMarkers                            = true;
                ScaleJustification                            = NinjaTrader.Gui.Chart.ScaleJustification.Right;
                //Disable this property if your indicator requires custom values that cumulate with each new market data event.
                //See Help Guide for additional information.
                IsSuspendedWhileInactive                    = true;
                Interval                    = 15; // Default calculation window in bars minimum is 2

                AddPlot(Brushes.Orange, "LOL");
                AddLine(Brushes.Snow, 0, "Zeroline");
            }
            else if (State == State.Configure)
            {
                 AddDataSeries(BarsPeriodType.Minute, Bars.BarsPeriod.Value); // Get data series
            }
            else if (State == State.DataLoaded)
            {
                // Debug - Chart Details
                Print("----- Chart Loaded -----");
                Print("Instrument: " + Instrument.FullName);
                Print("Bars loaded: " + Count);
                Print("Bar time period is: " + Bars.BarsPeriod.Value);
                Print("Bar period type: " +  Bars.BarsPeriod.BarsPeriodType);
                Print("------------------------");
            }

        } // End of OnStateChange block

        protected override void OnBarUpdate()
        {
            myCount = (Count - 2); // Total bars, accounting for zero index and 'building' bar

            if (CurrentBar < myCount) // Waiting until whole chart is loaded...
            {
                Value[0] = 0; // Indicator value zero if loading bars
                return;
            }
            else            
            {    
                if(CurrentBar == myCount) // Fully loaded? Init indexes
                {
                    Print("Current Bar Index is: " + CurrentBar + " Initializing...");
                    startIndex = CurrentBar;
                    endIndex = Math.Abs(CurrentBar - (Interval - 1));
                }

                for (int barsAgo = CurrentBar; barsAgo >= 0; barsAgo--) // Driving loop
                {
                    if(endIndex >= 0)
                    {
                        Print("Starting index is: " + barsAgo + " Time: " + Time[barsAgo] + " Close: " + Close[barsAgo]);
                        Print("Ending Index for interval is: " + endIndex + " End Time: " + Time[endIndex]);
                        endIndex = endIndex - 1; // Decrement end

                        // Inside this for loop we do our interval processing - using barsAgo/end indexes
                        //
                        for(int i = barsAgo; i > endIndex; i--)
                        {
                            // Do indicator calcs in here for each interval...

                            Print("Interval process -- Index: " + i); // Debug
                        }
​
                    }
                }

                // Debug
                Print("[--------------------------------------------]");
            } // End of CurrentBar < (Count - 2) else block
        } // End of OnBarUpdate() block

// -------------------------- Anything Below Is Generated - Don't Touch!! ----------------------------------------

        #region Properties
        [NinjaScriptProperty]
        [Range(2, int.MaxValue)]
        [Display(Name="Interval", Description="Interval window in price bars", Order=1, GroupName="Parameters")]
        public int Interval
        { get; set; }

        [Browsable(false)]
        [XmlIgnore]
        public Series<double> LOL
        {
            get { return Values[0]; }
        }

        #endregion

    }
}

#region NinjaScript generated code. Neither change nor remove.

namespace NinjaTrader.NinjaScript.Indicators
{
    public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase
    {
        private TTBullBear[] cacheTTBullBear;
        public TTBullBear TTBullBear(int interval)
        {
            return TTBullBear(Input, interval);
        }

        public TTBullBear TTBullBear(ISeries<double> input, int interval)
        {
            if (cacheTTBullBear != null)
                for (int idx = 0; idx < cacheTTBullBear.Length; idx++)
                    if (cacheTTBullBear[idx] != null && cacheTTBullBear[idx].Interval == interval && cacheTTBullBear[idx].EqualsInput(input))
                        return cacheTTBullBear[idx];
            return CacheIndicator<TTBullBear>(new TTBullBear(){ Interval = interval }, input, ref cacheTTBullBear);
        }
    }
}

namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns
{
    public partial class MarketAnalyzerColumn : MarketAnalyzerColumnBase
    {
        public Indicators.TTBullBear TTBullBear(int interval)
        {
            return indicator.TTBullBear(Input, interval);
        }

        public Indicators.TTBullBear TTBullBear(ISeries<double> input , int interval)
        {
            return indicator.TTBullBear(input, interval);
        }
    }
}

namespace NinjaTrader.NinjaScript.Strategies
{
    public partial class Strategy : NinjaTrader.Gui.NinjaScript.StrategyRenderBase
    {
        public Indicators.TTBullBear TTBullBear(int interval)
        {
            return indicator.TTBullBear(Input, interval);
        }

        public Indicators.TTBullBear TTBullBear(ISeries<double> input , int interval)
        {
            return indicator.TTBullBear(input, interval);
        }
    }
}

#endregion

And for the life of me --- I can't understand why it still runs TWICE. I tried to compare it to the vanilla "Bars With Currant Glaze" Indicator I made earlier, but I don't see any obvious differences.

Any idea @bitdavid what is going on? Its totally baffling to me.

But at least the sliding window thing works. Starts at the oldest bar, and calculates a start/end index that can be fed into a FOR loop for further processing.

Appreciate any input, as it helps me and I am willing to share the 'skeleton' code to help others out.