Strategy Testing - Indicator logic being skipped?

[RDE_other]

Hello,

I am building a strategy strictly using the NT8 builder.

In back testing I have noticed the strategy entering positions the logic would not allow it to enter.

In the attached image both "buy" positions should not have been placed.

Neither the first or second buy pass the BOP(0) > 0.005. (pictured) I have attempted this with other values (0, .05, .01. +-, etc.) and it will still enter in this position.

Within the "set" it is "if all".

I have had the same issue with a "cross above" for another indicator. It did not cross above but stayed above within the period defined.

0) What could cause a lapse in checking conditions?
a) Could a race condition be occurring between the threads/ram/output? If yes how would I eliminate or test this?

1) Is there a limit to how many conditions may be present within a set?

2) If using cross above with an indicator in the left and right does it check and compare:
a) Same index position on both sizes [y(3) vs x(3) then y(2) vs x(2) then....]?
b) Left position back compared to current on right [y(3) vs x(0) then y(2) vs x(0) then...]?
c) Compare all left vs all right as in a matrix [y(3) vs x(3)...x(0) then y(2) vs x(3)...x(0) then...]?

3) Can comparisons above/below/cross to either an int or double of 0 cause divide by zero issues?
a) If yes what is the most reasonable approximately zero number to use?

4) Can a cross above/below check fail and become a simple greater than or less than?

5) Despite choosing the fine and per tick testing (as well as per tick calculation) I have noticed that in testing the strategy always waits for the next candle to enter a position. Is this standard behavior and can it be overridden?

[NinjaTrader_Gaby]

Hello RDE_other,

Thank you for your post.

There is no limit to how many conditions you can put in a set.

The way the cross above evaluates is it that checks the number of bars specified in the lookback period and sees if the a cross happened within those last x bars.

For example:

// Go long if 10 EMA crosses above 20 EMA within the last bar

if (CrossAbove(EMA(10), EMA(20), 1))

EnterLong();

Can you explain further what you mean by "Can comparisons above/below/cross to either an int or double of 0 cause divide by zero issues?" If you mean specifying a lookbackPeriod of 0, this would cause issues because you are saying the method should look within the last 0 bars to see if a cross occurred.

"Can a cross above/below check fail and become a simple greater than or less than?"

Can you explain what you mean by fail? Checking whether a cross happened will return either a true or false value (either the cross happened or it did not).


If your strategy is set to Calculate.OnEachTick, you will need to implement intrabar granularity and enable Tick Replay for higher accuracy during backtest. In order to implement intrabar granularity, you will need to unlock your script as it is not possible to implement in the Strategy Builder. Please see this forum post for a detailed explanation:

https://forum.ninjatrader.com/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live

As far the logic in your strategy not executing, are there errors appearing on the Log tab of the Control Center?

Try testing the Sample MA Crossover strategy included with NinjaTrader on the same chart or Strategy Analyzer using the same instrument, bar type, interval, and date range.

Do you see results with the Sample MA Crossover strategy? (This would confirm for us data is available and the issue is with the logic in the custom script)


If the strategy is getting data and the strategy is enabled or a backtest is run with no errors in the Log tab of the Control Center, and the Sample MA Crossover is returning results, then would likely indicate the logic conditions in the custom strategy did not evaluate as true or orders are being ignored or cancelled.

In order to better understand how the code is working, it will be necessary to use Print to see how the conditions are evaluating and enable TraceOrders to see if orders are being submitted, ignored, rejected, or cancelled.

Below is a link to a forum post that demonstrates using prints to understand behavior and includes a link to a video recorded using the Strategy Builder to add prints.

https://ninjatrader.com/support/forum/forum/ninjatrader-7/strategy-development-aa/99209-date-conditions-not-working?p=791121#post791121

Enable TraceOrders, print the time of the bar and all values used in the conditions that submit entry orders. Include labels for all values and comparison operators.

Let me know if you need any assistance creating a print or enabling TraceOrders.

Save the output from the output window to a text file and provide this with your reply.

I'll be happy to assist with analyzing the output.

[RDE_other]

Hello, NinjaTrader_Gaby,

Thank you for the reply.

Firstly, yes, the sample MA cross over works with the specific market, time frame, date, fineness, and tick.

Neither the sample MA nor the strategy built code generated any error logs. The log is working as there are several messages related to simulation connection, session break, and the like.

-------------

I used the strategy builder print() link that ChelseaB provided in the second link you provided to get some debugging done more quickly than not at all.

Following that video I added a set which prints different outputs. At this point in time I have not unlocked any code. I am unable to unlock and custom code at this time.

Attached shows the outputs at the appropriate time, the chart, the graphs, and highlights which pinpoint some of the oddities.

-------------

The set (if statement) which has the print() is the last in the code.

In the first picture there is an apparent mismatch between the BOP graph and output. The graph showing negative at the timestamp and the output showing positive.

Additionally, there is the odd nature of the BOP output having repeat values.

I can make a few assumptions but the short of the long is the graph is different than the chart at a minimum.

In the second picture the BOP mismatch appears to remain along with the repeat values.

For a long to be ordered one of the conditions to be satisfied is BOP[0] > BOPshortLONG. In this instance ~0 < 0.005

-------------

To my prior questions:

Regarding divide by zero:
I am asking whether have an int or double equal 0 can cause issues? I then proffered a possible divide by zero error.

Regarding crossover:
I am asking how crossing is determined. The actual mechanics of the check. Whether it is checking a matrix of possibilities, a stepped index comparison, or a comparison between the first indicator at the different index locations against the second indicator at index[0]?
In addition to the above I was asking about a how the comparison of the crossover worked. In particular whether it called other operations such as great than?

[NinjaTrader_Gaby]

Hello,

Thank you for your response.

It is a bit unclear from the output what comparisons your script is making. For example, you mention one of the conditions is BOP[0] > BOPshortLONG. It would be helpful if this was illustrated in the output so we can how the conditions are evaluating and why certain logic is being ignored.

The prints should include the time of the bar and should print all values from all variables and all hard coded values in all conditions that must evaluate as true for this action to be triggered. It is very important to include a text label for each value and for each comparison operator in the print to understand what is being compared in the condition sets.

Additionally, please save the output from the output window (right-click within the output window > Save As) to a text file and provide this with your reply.

"I am asking whether have an int or double equal 0 can cause issues?" In what scenario? Yes, dividing by zero in any case (using an int or double) will cause an error.

A cross is determined as below:

if (SeriesA[1] < SeriesB[1] && SeriesA[0] > SeriesB[0])
// cross above
​

Essentially, this comparison is being done but being looped through the specified lookBackPeriod.

[RDE_other]

Hello, NinjaTrader_Gaby,

Thank you for the reply.

I appreciate the follow up on my initial questions.

I'm quite concerned with the obvious chart/graph/output discrepancies that I highlighted and the strange duplicate outputs across multiple timestamps.

Being blunt I'm unsure how more output in a semi-specific format without the code is going to solve this.

If you really just want output I'll work on getting it to your liking this weekend. Otherwise I have no problem providing the code and how to replicate/find the apparent error.

[NinjaTrader_Gaby]

Hello,

Can you please provide a screenshot of the Data Box open with your mouse placed over the bar? This is so we can compare the Data Box's data with the timestamped bar info that is being printed to the output.

Data Box - https://ninjatrader.com/support/help...8/data_box.htm

Can you also also provide a screenshot of your print statement, so we can see exactly what is being printed?

Thank you in advance.

[RDE_other]

Hello, NinjaTrader_Gaby,

Thank you for the reply.

Attached are pictures of the 11th and 16th ES MAR24 with this particular script. Additionally, the next post will include the the print() outputs, long logic, and initial conditions.

I took screenshots of the buy tick and the one immediately preceding it with output, chart, graphs, databox, conditions all present.

Note that I always change the BOPlongCROSS and BOPshortCROSS initial conditions from default to what is seen in the appropriate user inputs box.

The pictures of the 11th show the program not following the logic.

The picture of the 16th when compared to the prior 16th pictures show everything offset by one tick now. I'm unsure why this may be. Depending on how the code runs in the background it may actually be following the script properly.

[RDE_other]

Issue with uploading pictures. Redoing them.

In playing with the script and getting rid of the BOP condition another condition stopped being read. I'm wondering if there is possibly an issue with my install. I'll explore that this weekend as well.

[RDE_other]

16th images.

[RDE_other]

Reinstalled with newest NT8 and no change in outcomes. Found another script which also appears to not follow logic checks.

[RDE_other]

Reimaged my HD and the logic appears to be working now. Not sure what made the difference or changed.

Two oddities remain:
Time offset by 3 hours (I assume it's a setting).
Data reporting different price for same bar (also setting?).

Either way, the thing I'm mostly concerned with, logic implementation, appears fixed.

[NinjaTrader_Gaby]

Hello,

I am not seeing a significant difference in the values for the BOP from the Data Box and what your script is printing out from your screenshots.


1/11/24 13:33:00 - Data Box: 0, Output: -3.806-E16 (rounded to 0 in the Data Box).

1/11/24 13:33:15 - Data Box: 0, Output: -3.806-E16 (rounded to 0 in the Data Box).

1/16/24 12:48:00 - Data Box: 0.0798, Output: 0.079761..

1/16/24 12:48:15 - Data Box: -0.00952, Output: -0.0095238..


The last image you posted that demonstrates the difference in price has the time stamp on the Data Box for the bar covered by the top of the yellow box so I wasn't able to compare this one.