Index out of range errors

[raintree]

Yes, the method is designed to show where there was no return from the processor to the strategy/indicator. It can happen on any of the index accesses. In this case, as you see, it seems to have been trapped on the first "obviously correct" (yeah, right) statement.

Normally, wherever we cannot get to the "after" statement tells us where the error is, as essentially that is saying that the indicator/strategy never returned from the call to the processor. That is why we would expect to see some indication of the error. That there is no indication of the error in the normal NT places, seems to imply that the strategy spun so out of control that it could not even return information from the processor to NT.

As I seem to never be able to step through an NT strategy bar-by-bar, in such intransigent cases, I have ended up recoding the strategy as an indicator, then stepping through it bar-by-bar until I trigger the error.

Ah. Thanks for the painstaking explanation. That really helps my understanding of what's going on inside the scattershot method.

Obviously, there are many gaps in my understanding of Ninjascript programming, some at a very elementary level no doubt, due to my lack of systematic training in the fundamentals of any programming language. I've learned all my meager programming knowledge from trial and error with the Ninja platform and the kindness of mentors like you in the forum. For example, your mention of stepping through an NT strategy bar-by-bar doesn't correspond to anything I'm familiar with in NT, but it does ring some bells about other things I've heard of but not used, like Visual Studio .NET debugger. Is that what you're referring to? Your bar-by-bar indicator workaround sounds ingenious, but for educational purposes would you suggest that exposure to Visual Studio or some other C# debugging tool is something I might benefit from?

I'll respond to your later post in the my next post.

[raintree]

I just looked into the .cs file, and found this snippet in OnBarUpdate()

            PrintWithTimeStamp("before cp1");
            if(CurrentBars[0] < 8650 || CurrentBars[1] < 40)
            PrintWithTimeStamp("after cp1");
                return;

Given that sequence, I would expect you to never process anything as the return; statement will always be executed.

The PrintWithTimeStamp("after cp1"); statement will be executed throughout your initial bars escape period, then should not print.

Methinks that the strategy is doing what you asked it to do., not what you intend it to do? You probably want to switch the 2 statements.

You are right on all counts! Switching the two statements worked. Now I get the error a few days later. More importantly, now that the method is working it indicates that the next suspect is probably cp4, i.e. the statement on line 82 :

int beginBarsAgo = CurrentBar - Bars.GetBar(new DateTime(Time[0].Year, Time[0].Month, Time[0].Day -1, beginHour, beginMin, 0));

Changing the Print sensor to PrintWithTimeStamp shows that this error occured at 2:00 AM, i.e.on the Close of the bar on which the whole set of barsAgo calculations for the timeRange measurements is computed. So the problem here has nothing to do with the previous suspect (i.e. the daily TF ATR(14) calc), which is a relief because it possibly means that I've at least got that part of the strategy right!


Also, this suspect is more interesting! The Time[0].Day - 1 variation on the normal DateTime syntax is an attempt to deal with the problem of getting data about a timeRange that spans the midnight hour (18:00 - 2:00 in this case). It took me YEARS to find the workaround for this, since the indicators and strategies using the TimeRange concept turn into pumpkins at midnight and start muttering phrases like "your ending time must be later than your beginning time." The solution was found here in a reference sample that showed that you can simply subtract a day from the Time[0].Day term in the full DateTime expression. http://www.ninjatrader.com/support/forum/showthread.php?t=19176

I don't know why that doesn't work here.

[koganam]

int beginBarsAgo = CurrentBar - Bars.GetBar(new DateTime(Time[0].Year, Time[0].Month, Time[0].Day -1, beginHour, beginMin, 0));

With your new skills, handling that one should now be child's play?

Try this, See what it tells you. I did not read the whole file: I stopped when I saw that very first error. Sorry.

int beginBar = Bars.GetBar(new DateTime(Time[0].Year, Time[0].Month, Time[0].Day -1, beginHour, beginMin, 0));
PrintWithTimeStamp("Begin bar: " + beginBar);
int beginBarsAgo = CurrentBar - beginBar;

[raintree]

Thanks for the new input. I've got to go now, so I'll sleep on this and get back to you tomorrow. (It seems like I do my best thinking while asleep these days.)

[raintree]

This is addressed first to you, koganam, who have been kind enough to help me with this issue, but I've noticed that this thread has attracted additional visitors (possibly indicating a lot of experience with index out of range errors),so I'll try to recapitulate what's been happening and comment more broadly in case others want to join in and help clarify the issues.

I've been going over this again and again using koganam's “scattershot” debugging method of sandwitching all indexed statements between “before cp#” and “after cp#” Print statements to trap the location of the index error. (See post #10 for a description of the method.) It's interesting to learn about this clever method but so far it doesn't seem to be resolving my original code problem. Possibly I am not interpreting the indications from the method correctly, but the Output file print traps seem to be giving me inconsistent information about the location of errors. Let me illustrate this with some screenshots. (Incidentally, I've copied the Output screento a word processing file so I can search it. It's 3483 pages long in the word processor version!)

Screenshot #1 from the end of the Output window suggests that there are NO problems with cp1 or cp2 because both generate both “before” and “after” prints and that the “culprit” may be cp3a because it produces no “after” print. Ok, that's fairly straightforward.

Screenshot #2 from the beginning of the Output window, however, suggests that there ARE problems with cp1 because there are no “after” prints from this part of the window.

So which is it? Is the cp1 statement a cause of the error or is it NOT?

It gets worse. This same kind of inconsistency occurs a many other places in the Output window record but these examples are difficult to capture with a screenshot because their distribution appears to be almost random. I say “almost” because actually there may be a pattern but it's difficult to detect because the timestamp of the Output prints jumps around randomly! (This can be seen in Screenshot #2.) I have no idea why this is happening!

Anyway, I now find myself spending more time in trying to understand (debug?) the debugging method than in debugging the original code. So I come back again to my unanswered question from post #16:

Would it be helpful to invest some time in learning to use a professional debugging tool like Visual Studio to debug a problem like this? Anyone?

Personally, I tend to think this might be a good investment of time at this stage of my learning curve with Ninjascript but I'd like to get the advice of others.

Of course, the SHORT CUT might be for Betrand to come back in, or one of the other Ninja staff, and look at more than just part of my original code (post #3) and point out the error of my ways! LOL. (Incidentally this code is not just pulled out of the air but is actually an attempt to adapt the relevant hints given in the Reference Samples).

Thanks again for any help past and future.

[koganam]

Screenshot1 clearly tells you that the statement it is trapping is defective, and why. The next step is to look at what that statement is doing and figure out what you are doing wrong over there. I believe that I even gave you a little snippet that breaks down that code, in order to see what it is generating, and whether it is in line with expectations. Whenever you trap an error, you need to resolve it.

As to your question about learning Visual Studio, I can only tell you that MY learning curve was very steep, and learning to use the debugger was even steeper. The advantage of having a whiz-bang debugger is the finer control that it gives you, but you still have to know what you are looking for. I know how to use the debugger, but I can tell you that I have written some pretty complex scripts and never needed to pull out the debugger, because using "Print()s" to debug, along with the NT messages, has always pinpointed the problem clearly enough for me not to need the VS debugger.

NT is a closed framework based on C#, so knowing C# does allow one more room to roam, but not many of my scripts need anything outside the NT framework, and if your purpose is to write NT scripts, my experience is that understanding the NT framework is the correct first step, because it hides a lot of the complexity of C#, taking care of things within the framework in the background, so to speak, while exposing the simpler parts that are the most useful for writing trading tools. One can then use general C# to enhance one's scripts and abilities.

Just my $0.02. YMMV.

[raintree]

Hi koganam. Thanks again for coming back. You comments about the value (or not) of learning Visual Studio sound like sage advice. It really helps to hear someone put in in perspective like that.

About the screenshots, yes I can see that Screenshot1 trapped an error from the cp3a statement and definitely told me to look closely at that. But it also did NOT trap an error from cp1. Whereas Screenshot2 DID show an error being trapped at cp1. These screenshots are from the same backtest run. Can you see my confusion about this double message? I don't understand how there can both BE and NOT BE an error in cp1. Sorry to be such a pain about this, but its a real question for me. Is there something I should know about giving different weight to false positives vs. false negatives from this method?

I've got to be away from my computer for a while now but I hope I haven't overtaxed your patience and I'll be keen to get back and see if there's an answer. And, again, thanks for your help.

[koganam]

Hi koganam. Thanks again for coming back. You comments about the value (or not) of learning Visual Studio sound like sage advice. It really helps to hear someone put in in perspective like that.

About the screenshots, yes I can see that Screenshot1 trapped an error from the cp3a statement and definitely told me to look closely at that. But it also did NOT trap an error from cp1. Whereas Screenshot2 DID show an error being trapped at cp1. These screenshots are from the same backtest run. Can you see my confusion about this double message? I don't understand how there can both BE and NOT BE an error in cp1. Sorry to be such a pain about this, but its a real question for me. Is there something I should know about giving different weight to false positives vs. false negatives from this method?

I've got to be away from my computer for a while now but I hope I haven't overtaxed your patience and I'll be keen to get back and see if there's an answer. And, again, thanks for your help.

Not quite.

Your output show only one definite error at cp3. All that cp1 shows is a probable failure to return. The correct procedure is to resolve the definite error, rather than speculate about a potential error. IOW, a failure to return may even be correct: it all depends on how the code is supposed to work, as well as how it works, so as long as you have a definite error, chasing other issues that have not yet spit out an error is "chasing the wind", and just serves to confuse issues.

No matter what, your errors must be resolved, in a serial fashion, one at a time, starting with one error. You have to start somewhere. Why not resolve the definite error first, then if you still have unexpected behavior, you investigate that?

Another thing that may help is that you should always make clean debug runs. In other words clear the output window before you rerun the code. That way, you always have output form a single run, so that you can see exactly what happens without being distracted by data from earlier runs.

[raintree]

Ok, that makes a lot of sense. The rules are clear to me now. Focu on definite errors and work one error at a time. Thanks for all the explanations.