Collection Of Historical data inside a strategy

[Nyman]

What's the best approach for collecting historical data of close, cumulative delta, and values of other indicators inside a strategy and storing their values in a jagged array? The example below doesn't execute at the for loop raising the following error "Error on calling 'OnStateChange' method: Object reference not set to an instance of an object."

private double[][] LoadInputData()
        {
            double [][] dataArray = new double [Bars.Count-1-predictionPeriod][];
            int dataLen = Bars.Count-1-predictionPeriod;
            Print("Number of Historical "+Bars.BarsPeriod+" Bars: "+Bars.Count);
            for (int i=0; i < dataLen; i++)
            {
                dataArray[i] = new double[] {
                        Close.GetValueAt(i),
                        Volume.GetValueAt(i),
                        cd.DeltaClose.GetValueAt(i),
                        mfi.GetValueAt(i),
                        sRSI.GetValueAt(i),
                        tsf.GetValueAt(i),
                        cci.GetValueAt(i),
                        adx.GetValueAt(i),
                        fRSI.GetValueAt(i),
                        atr.GetValueAt(i)
                } ;
            }

            Print("JaggedArray Input data load finished: "+dataArray);
            // Create a ZScore filter to normalize the input data
            Print("Begin data normalization");
            double[][] inputsNormalized = Accord.Statistics.Tools.ZScores(dataArray);
            Print("Completed data normalization");
            Print("Input Data loaded return");
            return inputsNormalized;
        }​

[NinjaTrader_Jesse]

Hello Nyman,

object reference not set to an instance of an object is a general C# error meaning an object you tried to use was null. You would need to find out what object is null if you wanted to work on that code further.

You can get data by using the GetValueAt method and supplying an index. GetValueAt is a method on any Series type object.

I would suggest to simplify what you are doing in the loop to find what is null and then expand 1 item and retest each time you make a change to find what is having a problem.

[Nyman]

Jesse, thanks again for your support. Indeed there is an issue in the for loop in particular when I try to get the cumulative delta close value at the index. I really want to capture these values too. My primary is a 10min series and my secondary is 1 tick series. Where have I gone wrong, help, please.

My code snippets below:

else if (State == State.Configure)
            {
                //secondary BarsArray[1]
                AddDataSeries(Data.BarsPeriodType.Tick, 1);
                Print("State Configured");
            }
            else if (State == State.DataLoaded)
            {
                //Initialize the indicators
                cd = OrderFlowCumulativeDelta(BarsArray[1],CumulativeDeltaType.BidAsk, CumulativeDeltaPeriod.Session, 0);
                mfi = MFI(20);
                sRSI = RSI(14,3);
                tsf = TSF(predictionPeriod, 20);
                cci = CCI(20);
                adx = ADX(20);
                fRSI = RSI(9,3);
                atr = ATR(predictionPeriod);​
           }

if (BarsInProgress == 0)
            {
                // Print the close of the cumulative delta bar with a delta type of Bid Ask and with a Session period
                deltaClose = cd.DeltaClose[0];

            }
            else if (BarsInProgress == 1)
            {
                // We have to update the secondary series of the hosted indicator to make sure the values we get in BarsInProgress == 0 are in sync
                cd.Update(cd.BarsArray[1].Count - 1, 1);
            }​

[NinjaTrader_Jesse]

Hello Nyman,

Are you unable to use GetValueAt with that plot?

Print(cd.DeltaClose.GetvalueAt(index));

The loop you are using would require using GetValueAt. If you don't use a loop and just process the data per bar you could use cd.DeltaClose[0]

[Nyman]

NinjaTrader_Jesse, Hi jesse, i didn't print to see what was returned. I simply commentted that part and the loop worked.

[NinjaTrader_Jesse]

Hello Nyman,

Your original post was about a null error, if that line was the cause of the error that means the variable you are using is null. The variable cd would need to be populated to avoid getting a null error.

I would suggest using a print to find out if you can get a value from that or if you continue to get a null error, if you continue to get a null error you need to check where you defined the variable and that it is being assigned.

[Nyman]

NinjaTrader_Jesse, I did try to Print that value but got the same null error each time. And Also the other indicators were returning close price values instead of their own values.

[NinjaTrader_Jesse]

Hello Nyman,

I would suggest to try and simplify the the test you are doing. In your first post you are using a custom method and we can't see where that is being called from. Depending on where that is being called from that may relate to the problem. I would suggest to remove that method and make sure your logic is in OnBarUpdate inside of your if (BarsInProgress == 0) condition.

[Nyman]

NinjaTrader_Jesse, Hello Jesse, Below is the complete code. The function for collecting historical data is being called in the State.DataLoaded and I'm yet to do anything in the BarsInProgress == 0 logic because it pointless. When I enable the strategy, I would like to collect historical values of the close price and values of all the indicators and save them in a jagged array. The collection should be done in the State.DataLoaded logic

#region Using declarations
using System;
using System.IO;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.Indicators;
using NinjaTrader.NinjaScript.DrawingTools;
#endregion

//This namespace holds Strategies in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Strategies
{
    public class MLRandomF : Strategy
    {

        private double deltaClose;
        private OrderFlowCumulativeDelta cd;
        private MFI mfi;
        private RSI sRSI;
        private TSF tsf;
        private CCI cci;
        private ADX adx;
        private RSI fRSI;
        private ATR atr;


        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Enter the description for your new custom Strategy here.";
                Name                                        = "MLRandomForestClassification";
                Calculate                                    = Calculate.OnBarClose;
                EntriesPerDirection                            = 1;
                EntryHandling                                = EntryHandling.AllEntries;
                IsExitOnSessionCloseStrategy                = true;
                ExitOnSessionCloseSeconds                    = 30;
                IsFillLimitOnTouch                            = false;
                MaximumBarsLookBack                            = MaximumBarsLookBack.TwoHundredFiftySix;
                OrderFillResolution                            = OrderFillResolution.Standard;
                Slippage                                    = 0;
                StartBehavior                                = StartBehavior.WaitUntilFlat;
                TimeInForce                                    = TimeInForce.Gtc;
                TraceOrders                                    = false;
                RealtimeErrorHandling                        = RealtimeErrorHandling.StopCancelClose;
                StopTargetHandling                            = StopTargetHandling.PerEntryExecution;
                BarsRequiredToTrade                            = 20;
                // Disable this property for performance gains in Strategy Analyzer optimizations
                // See the Help Guide for additional information
                IsInstantiatedOnEachOptimizationIteration    = true;
                predictionPeriod                             = 4;
                ForestSize                                     = 100;
                TradeAlert                                    = true;
                AlertSoundBull                                = @"alert1.wav";
                AlertSoundBear                                = @"alert4.wav";
            }
            else if (State == State.Configure)
            {
                //secondary BarsArray[1]
                AddDataSeries(Data.BarsPeriodType.Tick, 1);
                Print("State Configured");
            }
            else if (State == State.DataLoaded)
            {
                Print("State Dataload initialize indicators");
                //Initialize the indicators
                cd = OrderFlowCumulativeDelta(BarsArray[1],CumulativeDeltaType.BidAsk, CumulativeDeltaPeriod.Session, 0);
                mfi = MFI(BarsArray[0], 20);
                sRSI = RSI(BarsArray[0],14,3);
                tsf = TSF(BarsArray[0], predictionPeriod, 20);
                cci = CCI(BarsArray[0],20);
                adx = ADX(BarsArray[0],20);
                fRSI = RSI(BarsArray[0], 9,3);
                atr = ATR(BarsArray[0],predictionPeriod);
                AddChartIndicator(cd);
                AddChartIndicator(mfi);
                AddChartIndicator(sRSI);
                AddChartIndicator(tsf);
                AddChartIndicator(cci);
                AddChartIndicator(adx);
                AddChartIndicator(fRSI);
                AddChartIndicator(atr);

                Print("State Dataload indicator initialization complete");
                // Get historical data
                double [][] dataInputs = LoadInputData();


            }
        }

        protected override void OnBarUpdate()
        {
            //Add your custom strategy logic here.
            if (CurrentBars[0] < 1 && CurrentBars[1] < 1)
                return;


            if (BarsInProgress == 0)
            {
                // Print the close of the cumulative delta bar with a delta type of Bid Ask and with a Session period
                deltaClose = cd.DeltaClose[0];

            }
            else if (BarsInProgress == 1)
            {
                // We have to update the secondary series of the hosted indicator to make sure the values we get in BarsInProgress == 0 are in sync
                cd.Update(cd.BarsArray[1].Count - 1, 1);
            }



        }



        private double[][] LoadInputData()
        {
            double [][] dataArray = new double [BarsArray[0].Count-1-predictionPeriod][];
            int lookBackPeriod = BarsArray[0].Count-1-predictionPeriod;
            Print("Data length: "+lookBackPeriod);
            Print("Number of Historical "+BarsArray[0].BarsPeriod+" Bars: "+BarsArray[0].Count);
            for (int i=0; i < lookBackPeriod; i++)
            {
                Print(""+Close.GetValueAt(i));
                Print(""+Volume.GetValueAt(i));
                Print(""+cd.DeltaClose.GetValueAt(i));
                Print(""+mfi.Value.GetValueAt(i));
                Print(""+sRSI.Value.GetValueAt(i));
                Print(""+tsf.Value.GetValueAt(i));
                Print(""+cci.Value.GetValueAt(i));
                Print(""+adx.Value.GetValueAt(i));
                Print(""+atr.Value.GetValueAt(i));

                dataArray[i] = new double[] {
                        Close.GetValueAt(i),
                        Volume.GetValueAt(i),
                        cd.DeltaClose.GetValueAt(i),
                        mfi.Value.GetValueAt(i),
                        sRSI.Value.GetValueAt(i),
                        tsf.Value.GetValueAt(i),
                        cci.Value.GetValueAt(i),
                        adx.Value.GetValueAt(i),
                        fRSI.Value.GetValueAt(i),
                        atr.Value.GetValueAt(i)
                } ;
            }

            for (int j = 0; j < dataArray.Length; j++)
            {
                Print("Row index: "+j);
                for (int c=0; c < dataArray[j].Length; c++)
                {
                    Print(""+dataArray[j][c]);
                    Console.Write("{0} ", dataArray[j][c]);
                }
                Console.WriteLine();
            }


            Print("JaggedArray Input data load finished: "+dataArray);
            // Create a ZScore filter to normalize the input data
            Print("Begin data normalization");
            double[][] inputsNormalized = Accord.Statistics.Tools.ZScores(dataArray);
            Print("Completed data normalization");
            Print("Input Data loaded return");
            return inputsNormalized;
        }




        #region Properties
        [Range(1, int.MaxValue), NinjaScriptProperty]
        [Display(Name="predictionPeriod", Order=1, GroupName="Parameters")]
        public int predictionPeriod
        { get; set; }

        [Range(1, int.MaxValue), NinjaScriptProperty]
        [Display(Name="ForestSize", Order=2, GroupName="Parameters")]
        public int ForestSize
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="TradeAlert", Description="Trade Alerts", Order=3, GroupName="Parameters")]
        public bool TradeAlert
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="AlertSoundBull", Description="Sound File for Bullish Trade Alerts", Order=4, GroupName="Parameters")]
        public string AlertSoundBull
        { get; set; }

        [NinjaScriptProperty]
        [Display(Name="AlertSoundBear", Description="Sound File for Bearish Trade Alerts", Order=5, GroupName="Parameters")]
        public string AlertSoundBear
        { get; set; }
        #endregion

    }
}
​

[NinjaTrader_Jesse]

Hello Nyman,

You won't be able to call your method from OnStateChange because the cumulative delta and other indicators need to run first. You would need to use that code from OnBarUpdate. DataLoaded just means the price data loaded but no indicators have processed at that point.

[Nyman]

Jesse, thanks a lot for pointing this out. I will make changes to the code as per your suggestions and help.

[Nyman]

NinjaTrader_Jesse, Hello Jesse, I made some changes and everything work as expected now. Kudos to you.