Question about data access from OnRender()

[ETFVoyageur]

I set a value in OnBarUpdate():

priceFilter[0] = ModifiedOptimumEllipticFilter()[0];

Then I print it out two ways: "priceFilter[0]" and "priceFilter.GetValueAt(CurrentBar)". As expected, both show the same value, and the value is the correct one. Then I access it from OnRender() as:

 double val = priceFilter.GetValueAt( <the same index> );

The problem is that the value retrieved in OnRender() is bad -- nothing like what was stored. priceFilter.Count was as expected -- the same as the number of bars -- so all of the data should be present.

What am I missing? Is there something I need to be doing for OnRender() to retrieve the correct data??

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

"The problem is that the value retrieved in OnRender() is bad"

What value is bad?

The value from priceFilter[0]?
Have you used TriggerCustomEvent() to synchronize the series first before using this series value?

Or did you mean the value from priceFilter.GetValueAt()?
Are you using an absolute bar index and not a barsAgo index?

[ETFVoyageur]

What value is bad?

The value retrieved from priceFilter was not correct.

The value from priceFilter[0]?
Have you used TriggerCustomEvent() to synchronize the series first before using this series value?

I did not use the square brackets accessor, -- I used GetValueAt() -- so I did not think that anything needed to be synchronized.

Or did you mean the value from priceFilter.GetValueAt()?

Yes.

Are you using an absolute bar index and not a barsAgo index?

Yes. I am using an absolute bar index. I'm doing this as I iterate over the bars in OnRender() and using that index.

[ETFVoyageur]

As an example to ensure clarity:


priceFilter is a Series<double>

From OnBarUpdate(), priceFilter.GetValueAt(1328) returns 2.45694995546673 just after storing the value. priceFilter[0] returns the same thing at that time. That is the correct value.

From OnRender() when iterating over the visible bars, priceFilter.GetValueAt(1328) returns 24.0747654158552, which is the wrong value.

Is there something I need to be doing in OnRender() before trying to access the priceFilter series?

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

I've given this a test and I am not able to reproduce. The value from the series in OnRender() is the same as from OnBarUpdate().

Attached is the test script, and the output saved from the output window produced on my end.

[ETFVoyageur]

Chelsea,

Thank you for taking the time to try to reproduce the problem. I installed your script and I have mixed news:

• The print statement in OnBarUpdate() after the conditional return was not executing. Rather than investigate, I just added a print statement ahead of the conditional and it showed what you said -- that the value saved in OnBarUpdate() was retrieved in OnRender(). So far, so good.
• To make our examples more alike, I replaced your SMA with the one my script uses: ModifiedOptimumEllipticFilter. That, too, agreed with you -- namely that no problem was in evidence.

Then I printed the specific line that has been giving a problem (line 1328). I did not expect that to be any different, but much to my surprise it was. OnBarUpdate() showed the correct value:

OnBarUpdate CurrentBar=1328 priceFilter[0]=2.47843777298688 GetAt=2.47843777298688

However, the print statement I added in OnRender():

Print($"at 1328={priceFilter.GetValueAt(1328)}");

printed out:

at 1328=24.0747654158552

That surprised me very much. It illustrates my original question with minimal changes to your script.

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

"The print statement in OnBarUpdate() after the conditional return was not executing."

This script only runs in real-time in OnBarUpdate() because OnRender() only runs in real-time and having prints in historical would only be clutter. May I confirm you are receiving real-time data and the bars are updating on the chart?

I can assure you the unmodified example does print in OnBarUpdate() in real-time. I can provide a video if you would like.


"Then I printed the specific line that has been giving a problem (line 1328). I did not expect that to be any different, but much to my surprise it was. OnBarUpdate() showed the correct value:"

Perhaps it's something that you've changed in the script. With the script I've provided the only value it would be retrieving is from the series on the specific index.

An export of a reduced test script (that doesn't include any other code) would be helpful.

[ETFVoyageur]

I don't see that realtime has anything to do with it, but perhaps that is related to the problem. I'm using this indicator on an end-of-day daily chart. There is no real-time data involved, only historical data.

I just added State to my printouts and OnBarUpdate is running only in State=Historical.

OnRender() is running only in State=Realtime.

I just assumed that is the way NT works. Are you saying that there is a problem related to having only historical data?

[ETFVoyageur]

Here is the lightly modified version of your script. My changes are marked at the start of the line with "/**/" to make it easy to see them. They are:

• I commented out setting the value with the SMA and added setting the value with ModifiedOptimumEllipticFilter. I doubt that matters, but I wanted to match the original problem as much as possible.
• I added two Print() statements.

That's it ... as I said before, not much change. I realize that my print in OnBarUpdate() generates a lot of output. My print is ahead of the Realtime check because OnBarUpdate() is called exactly once, in State=Historical.

The output remains as I said earlier. OnBarUpdate() sets the correct value for bar 1328:

OnBarUpdate CurrentBar=1328  State=Historical  priceFilter[0]=2.47843777298688 GetAt=2.47843777298688​

OnRender() gets a different value when fetching that bar:

OnRender() State=Realtime  value at 1328=24.0747654158552​

Here is the script (I omitted the generated code at the bottom):

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Indicators in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Indicators
{
    public class AccessSeriesFromOnRenderTest : Indicator
    {
        private Series<double>    priceFilter;
        private SMA                mySMA;
        
        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Name            = "AccessSeriesFromOnRenderTest";
                Calculate        = Calculate.OnEachTick; // even though OnEachTick will use higher resources, it will let us see in the prints as the value is being changed
            }
            else if (State == State.DataLoaded)
            {
                ClearOutputWindow();

                priceFilter        = new Series<double>(this);
                mySMA            = SMA(1); // some indicator to get a value from
            }
        }

        protected override void OnBarUpdate()
        {
/**/            //priceFilter[0] = mySMA[0];
/**/            priceFilter[0] = ModifiedOptimumEllipticFilter()[0];

/**/            Print($"OnBarUpdate CurrentBar={CurrentBar}  State={State}  priceFilter[0]={priceFilter[0]} GetAt={priceFilter.GetValueAt(CurrentBar)}");

            if (CurrentBar < 1 || State != State.Realtime)
                return;
            
            Print(string.Format("{0} | OnBarUpdate | CurrentBar: {1}, priceFilter[0]: {2}, priceFilter[1]: {3}", Time[0], CurrentBar, priceFilter[0], priceFilter[1]));
        }

        protected override void OnRender(ChartControl chartControl, ChartScale chartScale)
        {
            base.OnRender(chartControl, chartScale);

            Print(string.Format("{0} | OnRender | BarIndex: {1}, priceFilter[0]: {2}, priceFilter[1]: {3}", Bars.GetTime(CurrentBar), CurrentBar, priceFilter.GetValueAt(CurrentBar), priceFilter.GetValueAt(CurrentBar - 1)));
/**/        Print($"OnRender() State={State}  value at 1328={priceFilter.GetValueAt(1328)}");
        }
    }
}
​

[ETFVoyageur]

FWIW: I doubt that it matters, but the chart I have been testing on is SOXL with a full-length data history.

[ETFVoyageur]

Here is an updated version of your test script. There is no change to the logic. I did modify my printing (yours remains untouched):

• I limited my OnBarChange() printing to just bar 1328 to eliminate the flood of output. Feel free to comment out the added "if(...)" if you want to see all the bars. I thought you'd appreciate seeing just the one line, though.
• I added a print in OnStateChange:State.Realtime. It shows the same wrong value for priceFilter bar 1328. That establishes that the problem is not due to being in OnRender().
• As before, my lines are marked with "/**/" at the start of the line.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Indicators in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Indicators
{
    public class AccessSeriesFromOnRenderTest : Indicator
    {
        private Series<double>    priceFilter;
        private SMA                mySMA;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Name = "AccessSeriesFromOnRenderTest";
                Calculate = Calculate.OnEachTick; // even though OnEachTick will use higher resources, it will let us see in the prints as the value is being changed
            }
            else if (State == State.DataLoaded)
            {
                ClearOutputWindow();

                priceFilter = new Series<double>(this);
                mySMA = SMA(1); // some indicator to get a value from
            }
/**/        else if (State == State.Realtime)
/**/        {
/**/            Print($"OnStateChange() State={State}  value at 1328={priceFilter.GetValueAt(1328)}");
/**/        }
        }

        protected override void OnBarUpdate()
        {
/**/            //priceFilter[0] = mySMA[0];
/**/            priceFilter[0] = ModifiedOptimumEllipticFilter()[0];

/**/            if (CurrentBar == 1328)
/**/                    Print($"OnBarUpdate CurrentBar={CurrentBar}  State={State}  priceFilter[0]={priceFilter[0]} GetAt={priceFilter.GetValueAt(CurrentBar)}");

            if (CurrentBar < 1 || State != State.Realtime)
                return;
            
            Print(string.Format("{0} | OnBarUpdate | CurrentBar: {1}, priceFilter[0]: {2}, priceFilter[1]: {3}", Time[0], CurrentBar, priceFilter[0], priceFilter[1]));
        }

        protected override void OnRender(ChartControl chartControl, ChartScale chartScale)
        {
            base.OnRender(chartControl, chartScale);

            Print(string.Format("{0} | OnRender | BarIndex: {1}, priceFilter[0]: {2}, priceFilter[1]: {3}", Bars.GetTime(CurrentBar), CurrentBar, priceFilter.GetValueAt(CurrentBar), priceFilter.GetValueAt(CurrentBar - 1)));
/**/        Print($"OnRender() State={State}  value at 1328={priceFilter.GetValueAt(1328)}");
        }
    }
}

#region NinjaScript generated code. Neither change nor remove.

namespace NinjaTrader.NinjaScript.Indicators
{
    public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase
    {
        private AccessSeriesFromOnRenderTest[] cacheAccessSeriesFromOnRenderTest;
        public AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest()
        {
            return AccessSeriesFromOnRenderTest(Input);
        }

        public AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest(ISeries<double> input)
        {
            if (cacheAccessSeriesFromOnRenderTest != null)
                for (int idx = 0; idx < cacheAccessSeriesFromOnRenderTest.Length; idx++)
                    if (cacheAccessSeriesFromOnRenderTest[idx] != null &&  cacheAccessSeriesFromOnRenderTest[idx].EqualsInput(input))
                        return cacheAccessSeriesFromOnRenderTest[idx];
            return CacheIndicator<AccessSeriesFromOnRenderTest>(new AccessSeriesFromOnRenderTest(), input, ref cacheAccessSeriesFromOnRenderTest);
        }
    }
}

namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns
{
    public partial class MarketAnalyzerColumn : MarketAnalyzerColumnBase
    {
        public Indicators.AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest()
        {
            return indicator.AccessSeriesFromOnRenderTest(Input);
        }

        public Indicators.AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest(ISeries<double> input )
        {
            return indicator.AccessSeriesFromOnRenderTest(input);
        }
    }
}

namespace NinjaTrader.NinjaScript.Strategies
{
    public partial class Strategy : NinjaTrader.Gui.NinjaScript.StrategyRenderBase
    {
        public Indicators.AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest()
        {
            return indicator.AccessSeriesFromOnRenderTest(Input);
        }

        public Indicators.AccessSeriesFromOnRenderTest AccessSeriesFromOnRenderTest(ISeries<double> input )
        {
            return indicator.AccessSeriesFromOnRenderTest(input);
        }
    }
}

#endregion

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

I won't be able to test this as it is not an export with the indicator as requested.

But on the same bar number, the original test script I provided is not able to reproduce, is this correct?

[ETFVoyageur]

I won't be able to test this as it is not an export with the indicator as requested.

Sorry ... I did not know that format was important ... here it is.

But on the same bar number, the original test script I provided is not able to reproduce, is this correct?

No, it is not. Your original script did not show enough data to know because, on my daily SOXL chart, OnBarUpdate() only got called in Historical so your print was never triggered. When the only thing I changed was to add a print statement that always printed, your original script DID show the problem.

To be clear, what I have been using IS your original script except for:

• It uses a different moving average, to match the one I was using when I found the problem.
• I added print statements to better illustrate the problem.

There is nothing magic about that specific bar. It's just the one where I first discovered the problem so I keep using it.

I doubt there is anything about SOXL. It's just the one where I discovered the problem so I keep using it because I know what values to expect.

There is nothing magic about the moving average. It happens even when OnBarUpdate() just adds CurrentBar instead of a moving average value. I have added a second export (...NoMA) showing that. That script gets:

OnBarUpdate CurrentBar=1328  State=Historical  priceFilter[0]=1328 GetAt=1328
OnStateChange() State=Realtime  value at 1328=3376​

[NinjaTrader_ChelseaB]

Hello ETFVoyageur,

For historical you would have to enable TickReplay for Calculate.OnEachTick to work.

https://ninjatrader.com/support/helpGuides/nt8/tick_replay.htm

Further, if you are going to be fetching a historical bar that is more than 256 bars ago, then you must use MaximumBarsLookBack.Infinite with the series.

https://ninjatrader.com/support/helpGuides/nt8/seriest.htm

https://ninjatrader.com/support/helpGuides/nt8/maximumbarslookback.htm

I've reduced the scripts and changed to OnBarUpdate() as you are now informing this only occurs with a historical bar and have changed to using MaximumBarsLookBack.Infinite and I am not able to reproduce.

[ETFVoyageur]

Thank you. That's the key. No more problem once I change to:

priceFilter = new Series<double>(this,MaximumBarsLookBack.Infinite);

Both the sample and my real code work fine with that change. I was hoping it was just something simple that I was overlooking.