SampleStreamReader

[zhiga]

Hi,

I am using SampleStreamWriter to write to a file. As a result, I get a txt file that looks like this:

12/21/2022 11:01:00 PM,178.3,178.3,171.6,176.6
12/21/2022 11:02:00 PM,178.6,178.6,175.6,172.4

Then, I'm using SampleStreamReader to get data from this file for further processing.
But I am getting this error:
System.FormatException: String was not recognized as a valid DateTime.
at NinjaTrader.NinjaScript.Indicators.SampleStreamRea der.OnBarUpdate()

So why this happened?

[NinjaTrader_Jesse]

Hello zhiga,

The error is that the date/time being parsed is invalid in some way.

Are there any extra lines at the top or bottom of the file or have you modified the file in any way?

To find out what the problem is specifically you can add a Print like the following to see what the last line of output is before the error.

if (splitCounter == 1)
{
    Print(s);
    date = ToDay(DateTime.Parse(s));
}

[zhiga]

Hello zhiga,

The error is that the date/time being parsed is invalid in some way.

Are there any extra lines at the top or bottom of the file or have you modified the file in any way?

To find out what the problem is specifically you can add a Print like the following to see what the last line of output is before the error.

if (splitCounter == 1)
{
Print(s);
date = ToDay(DateTime.Parse(s));
}

No extra lines. No modification.

The Print result is Date from first line and the number of something:

12/16/2022 11:01:00 PM
507638
System.FormatException: String was not recognized as a valid DateTime.
at NinjaTrader.NinjaScript.Indicators.SampleStreamRea der.OnBarUpdate()

[NinjaTrader_Jesse]

Hello zhiga,

From that output it looks like there is some error in the data. If the output was 2 lines:

12/16/2022 11:01:00 PM
507638

The second line is the print that happened before the error and 507638 is not a date time.

I would suggest opening a small timeframe like 1 day of 30 minute bars and then use the sample stream writer then re test opening the file with the stream reader. I tried using the two samples on my end just now and see it working as expected.

[zhiga]

Hello zhiga,

From that output it looks like there is some error in the data. If the output was 2 lines:

12/16/2022 11:01:00 PM
507638

The second line is the print that happened before the error and 507638 is not a date time.

I would suggest opening a small timeframe like 1 day of 30 minute bars and then use the sample stream writer then re test opening the file with the stream reader. I tried using the two samples on my end just now and see it working as expected.

Thank you Jesse,

I did everything all over again. Now the script is working.
I have another question:
how can I put data from string[] array in double[,] array?

[NinjaTrader_Jesse]

Hello zhiga,

Are you referring to the following line?

string[] split = line.Split(new char[] {','});

If so there are some examples of converting that string to a double a few lines down from that, You have to loop over the array to get each split individual string and then parse it. for example:

foreach (string s in split)
{
      splitCounter++;

      if (splitCounter == 1)
         date = ToDay(DateTime.Parse(s));

          if (ToDay(Time[0]) == date)
         {
                if (splitCounter == 2 && currentOpenSaved == false)
               {
                      currentOpen = double.Parse(s);

For a string to be a double the string would need to be a string of a number. If you have a number that is a string you can parse it into a different type like a double.

https://learn.microsoft.com/en-us/dotnet/api/system.double.parse?view=netframework-4.8.1

string myString = "2001.85";
double myDouble = double.Parse(myString);

[tdn18102]

Hi NinjaTrader Jesse,
I am trying to read data using "SampleStreamReader" but no success.
The following are 2 sets of my data:

1st set:

8/12/2024 9:45:00 AM,167.62,167.97,166.13,166.28,3878366
8/12/2024 10:00:00 AM,166.35,166.88,166.11,166.64,1670904

2nd set: Different Date Format

20240722 102700;958.19;959.4;956.8;959.4;2363
20240722 102800;958.57;962.0;958.26;961.79;3487
20240722 102900;960.05;961.9;960.05;961.11;1433

Both sets will getting error when using the: SampleStreamReader as follow:

System.FormatException: String was not recognized as a valid DateTime.
at NinjaTrader.NinjaScript.Indicators.SampleStreamRea der.OnBarUpdate()

Would you please provide me some helps and/or direction;

in addition, my data contain Volume Parameter at the end:
Date/time; Open; High; Low; Close; Volume.


Thank you very much

[NinjaTrader_Jesse]

Hello tdn18102,

Have you tested using both the streamwriter and stream reader examples from the help guide? That shows the correct way to write and also parse data from the csv file.

[tdn18102]

Hi Jesse, I've tested both the streamwriter and StreamReader examples from the help guide; the StreamWriter is working fine.
And the StreamWriter provided output format as below:

AMZN,8/12/2024 9:45:00 AM,167.62,167.97,166.13,166.28,3878366
AMZN,8/12/2024 10:00:00 AM,166.35,166.88,166.11,166.64,1670904
AMZN,8/12/2024 10:15:00 AM,166.92,167.87,166.83,167.57,1817132
AMZN,8/12/2024 10:30:00 AM,167.57,168.55,167.47,168.03,1529982
AMZN,8/12/2024 10:45:00 AM,168.04,168.51,167.7,168.01,1609207
AMZN,8/12/2024 11:00:00 AM,168.02,168.17,167.78,168.05,750539

The problem is: I do not know what format, and sequence that NinjaTrader required to parse information to import Data to "Historical Data"

If I keep Data Format as below under .txt (AMZN.txt), and Import data manually from Control Center ---> Import -->Historical Data: with the Below Data Format then everything OK working Fine.

20240726 093000;180.46;181.21;180.24;181.18;1457250
20240726 093100;181.22;181.39;180.63;180.87;119335
20240726 093200;180.89;181.22;180.79;181.12;174914
20240726 093300;181.1;181.25;180.61;180.63;168744
20240726 093400;180.63;180.98;180.59;180.86;113134

I am in a ROAD BLOCK, please help.

Thank you very much

tdn18102

[NinjaTrader_Jesse]

Hello tdn18102,

The two samples are intended to be used together directly, the streamwriter will write out data from the chart where it is applied as an example. The streamreader is intended to read that file specifically and not the historical data file you are using. You can see the difference in the two samples you provided, the writer puts the date in the format you see and has the various data points separated by commas. The file you are trying to read is not in that format so it cannot be used with the sample.

Those samples are really intended for you to learn the C# code and not be used directly for a project. When you read and parse data you are going to need to write your own code that matches that specific dataset so when each line is read and the string is split you can parse each part correctly.

If you are trying to import historical data you can use the historical data manager for that.

https://ninjatrader.com/support/helpGuides/nt8/importing.htm