MTF + out range index value + BarsRequiredToTrade : problem

[Amedeus]

Hey Chelsea, thanks,

is this enough to show why the chart is running ?

[NinjaTrader_ChelseaB]

Hello Amedeus,

Just a recommendation, I recommend keeping things simple. Remove all prints that are not directly what you are trying to see. Anything else becomes clutter that you have to read through which can cause confusion.

This print, by itself can cause an error:

Print(string.Format(" BarsRequiredToTrade = {0} | CrossAbove(Closes[2], KC2.Lower, 200) = {1} | CurrentBars[2] < 200 = {2}"
, BarsRequiredToTrade, CrossAbove(Closes[2], KC2.Lower, 200), CurrentBars[2] < 200 ));

It calls CrossAbove with a period of 200 without any check there are 200 bars.

The print I have suggested to you in post# 46 will not error. This does not attempt to call an invalid index.
Try printing the print I have suggested at the top of onbarupdate. Remove all of the other prints.

[Amedeus]

Hey Chelsea, thank you,

allright, here is the new test and its output :
one simple print and we keep the crossabove at the end.

I can see code and output lighter, but still can't relate the chart-analyzer divergence.

[NinjaTrader_ChelseaB]

Hello Amedeus,

So you are calling CrossAbove(Closes[2], KC2.Lower, 200) when CurrentBars[2] is 20. That will cause an error. You cannot call 200 bars ago when only 20 bars exist.

[Amedeus]

hey Chelsea, thanks,

So you are calling CrossAbove(Closes[2], KC2.Lower, 200) when CurrentBars[2] is 20. That will cause an error

Does the chart output not make this truth lie ?
How not ?

in test8 we can see the print of the crossabove as evaluated after the 20th CurrentBars[2] in the chart output.

[NinjaTrader_ChelseaB]

Hello Amedeus,

I'm not understanding the question.

CurrentBars[2] is 20 when the code is evaluated because BarsRequiredToTrade is 20..

The 200 you have used in CrossAbove is a number of bars. Its comparing to see if a cross has occurred anytime in the last 200 bars.

If CurrentBars[2] is 20 and is less than 200, then there are not 200 bars.
It's bar 20, what is 200 bars ago?

If you have 20 apples, how do you subtract 200 apples? You cannot subtract 200 apples when you only have 20 apples. Once you have 200 apples or more you can subtract 200 apples.

If CurrentBars[2] is greater than 200, then there are 200 bars that can be checked for a crossabove on.

[Amedeus]

Hey Chelsea, thank you,

I do totally understand what you say.
And am glad that I can agree with that kind of common sense, and you too.


From 20 Bars, the code starts to be evaluated.

then, as soon as the code starts to be evaluated, of course the analyzer finds 200 too much to find :
there was no check, and the biggest number that the analyzer knows for now is 65, which is the indicator call.

on the contrary, the chart keeps going like nothing.
It is fine for the chart to be asked a 200 barsAgo when it is evaluating the 21st, 22th and so on.
The chart is sick. the chart has trouble recognizing that 200 barsAgo is out of its range.
The chart should not be evaluating its own 21st or 22nd CurrentBars[2] : it should have stopped already.

Don't you agree ?

If the chart is a machine and cannot be sick, then I am missing something here.
How come are you not all glad and cheering that we've braught to the table a script that is able to lookback more than what it has previously checked, what a record !

[NinjaTrader_ChelseaB]

Hello Amedeus,

I would expect that if the script does hit an error it will stop processing.

As I am seeing this entire thread was never about correcting an index error, you were only asking why the strategy on the chart is not hitting an error when you expecting an error, I have created a script to test.

I'm not seeing an error on either the chart or the strategy analyzer.

https://drive.google.com/file/d/1MDYVxL1r9uNvHOj5hyW7zw_5LkSeBWDq/view?usp=drivesdk

To confirm, are you able to get an error in the strategy analyzer using this exported simplified test script?

[Amedeus]

Hey Chelsea, thank you,

Correcting the index error was part of clearly understanding the mechanics, but I find your description right.

In your CrossIndexErrorTest_NT8 , I am not seeing any error on both too.
There is no check but the strategy evaluates the 200 Ago.
And it has been made super clear that without a previous check, such a far away lookback should cause an error.




Also, if we add
if (BarsInProgress == 1 && CurrentBar == 0)
on my divergent strategy, it starts to be evaluated in both analyzer and chart.
But instead of stopping immediately like in your example video, it evaluates all the bars.
We can see that in the attached test9.txt, CurrentBar is not aqual to 0.
Please let me know if in this case we should expect CurrentBar to be greater than 0.

[NinjaTrader_ChelseaB]

Hello Amedeus,

Unfortunately, I don't know the internal code of CrossAbove/CrossBelow, so I can't say for sure if that would cause an indexing error. I do know that properly coding a script to never even attempt to use an invalid index is the proper way.

Have you reduced your code to only a single line that can reproduce the error in the strategy analyzer and not on the chart?

May I have the export so I that I can show this to our development?


To answer your question, if there is an action block for 'if (BarsInProgress == 1 && CurrentBar == 0)' I would expect this action block only only run if the condition evaluates as true and CurrentBar is 0.

[Amedeus]

Hey Chelsea, thank you very much,

I took the liberty to consider this thread not in a rush.


"never even attempt to use an invalid index" ?

If we discover this theory : there is a limit to what the code can do depending on the Lookback,
Avoiding the error causes the positive outcome. Consciously provoking the error is its reciprocity : the negative outcome.
The reciprocity, like with maths, is a confirmation of the theory, one other angle to it. This thread is born from the very simple need to verify and test the theory.
It's about full assimilation of this Major aspect of how the script does.
Once this is done, I would agree no one should dare to object such a "never".


**

We are like in situation posts #27 : after testing a healthy code and testing again the error causing line, the big lookback line hits an error on both.
This is good. But it does not explain at all how the switch in behavior.

I tried to reproduce the (divergent chart/analyzer output) error like I thaught I did with post #29's script : unsuccessfully .
"I thaught I did" because after impoerting the post #29 file (which was uploaded from us), it tested erroring on both chart and analyzer.
Unlike the previously reported outputs, from the time of the export. (post#29)
I assume that like around here, you can sense some inconsistency with that.

So, I guess there is nothing more to export than post #29 file ...
I could not say for sure what this inconsistency suggests, but it's apparently not from the code.

[NinjaTrader_Jim]

Hello Amedeues,

This is Jim responding on behalf of Chelsea who is out of the office at this time.

I question the value behind "why did the script generate an index out of range under X scenario versus Y scenario" when the ultimate reason for receiving the index out of range was because the script allowed accessing some indexes that would have been further back than the script would need. I understand the curiosity, but to me, this comes down to a clear rule that we should follow was broken. We should always make sure the script processes enough bars before we try to look back that many bars.

Using MaximumBarsLookBack.TwoHundredFiftySix results in the Series being built as a ring buffer. If we think of the script as a mathematical function, same input data + same code + same settings = same result. Testing on a different set of data at a different time will ultimately change the result and we may hit or miss the error you are questioning.

If a clearer answer is needed, I am willing to test the script from post #29 with a specific set of data and timezone (Market Replay in Playback can help ensure an apples to apples test,) but we will come back to say that since the KeltnerChannel uses 73 bars of the 540 tick data series, and the CrossAbove uses 300 bars of this data series, Our CurrentBar check for this data series should be 300 at a minimum.

[Amedeus]

Hey Jim, thank you very much,

It is not only about free curiosity, the original script has been evaluated as it did not have enough of a check for its Lookback.
This is what really needs to be broke down.
Until one day it started to error out (about time!).
"why did the script generate an index out of range under X scenario versus Y scenario" is just the 1st symptomatic question from the previous observation.
This question not beeing well oriented must relate to the level of expertise of who's expertizing.
Its value could reside in any type of correction. Especially any of those orienting back toward the cause behind the observed symptom.
What about just below after the stars ?

**

Testing on a different set of data at a different time will ultimately change the result and we may hit or miss the error you are questioning.

Are you saying that our script (anyone of them actually, without an exposed enough lookback) could eventually NOT hit an error, depending on the data set it's applied on ??

Such an impossible script - with the parameter MaximumBarsLookBack.TwoHundredFiftySix -
would be evaluating a wrong double from the series collection, due to the ring buffer nature of the MaximumBarsLookBack array ? Is that correct ?

For example : asking for BarsAgo 306 would make the script look at the 50th double behind the tail of the MaximumBarsLookBack array ? (would it then be the 49th behind the head too ?)
We would be left with a non-erroneous/evaluating script, that would evaluate a "fake" period (wrong array).
Could a our script behave as described ?

[NinjaTrader_Jim]

Hello Amedeus,

I mean that if you do not include a CurrentBar check and you try to reference a bars ago or look back further than how many bars the script has currently processed you can expect index out of range scenarios.

MaximumBarsLookback.TwoHundredFiftySix will make the Series a ring buffer, and if you try to reference beyond 256 bars in this series, you can expect an error or an unexpected value. If values are assigned to the Series on each bar and you are on bar 306 this would essentially be the 50th element in the array. However, Series objects are different from arrays in that their elements shift as new bars form. MaximumBarsLookback.Infinite should be used with Series<t> that need to hold more than 256 values, otherwise expect a garbage in/garbage out computation scenario.

If you have a specific question regarding a specific piece of code and specific set of data that we can test on our end with some steps, we could give more specific answers.

[Amedeus]

Hey Everyone,

Well, from what I can understand, going further with this , we'd be in the error zone by default anyaway.

Thank you very much Chelsea, Paul and Jim, it was so cool to discuss outputs and errors with you.
We definitely uncovered some very helpful NS basic functionng subtleties or reducing methodology. Within the great MTF context.
You guys rock.

Cheers