Backtesting renko charts invalid

[billr]

Hello.

I use renko charts for the majority of my forex/futures trading. Those of us that trade using non-time based charts tend to be systematic, technical traders and are particularly reliant on accurate information and analysis.

I realize that NT's backtesting engine is invalid for renko charts. Nevertheless, while it seems silly, on more that one occasion I have set aside the shortcomings of the NT backtesting engine when focused on mechanical trading system development and let myself go down a dead end path of invalid strategy testing. The frustration resulting from this waste of time and energy is hard to overstate. Even so, I can't seem to convince myself to abandon the idea of trying to use NT to develop, backtest and trade renko based trading strategies.

Therefore, I am considering investing time and money into developing a work-around. But before I do, I would like to know if there is more to the problem that what I have observed. Namely, most of the logic flaws seem to relate to the universal practice of programmers assuming it is perfectly fine to ignore the data outside the renko block. The trader uses this chart style to filter market noise, the backtest engine logic cannot and remain valid. For example:

1. Fill logic is flawed. It is assumed by the backtest logic that fill price of the open of the next bar is equivalent to the Close of the already closed bar that triggered a trade. (The folks at Tradestation built in this exact same error.)

2. The fact that all data outside the bars is deleted or ignored also renders results for
stop losses, trailing stops and profit targets invalid. (if my assumptions are correct in how the backtest logic works). Another crazy flaw seen at Tradestation.

Could you comment on whether the backtesting logic has other flaws that are not a result of the data deletion problem?

Also, I am curious why the backtesting logic could not be fixed to solve this problem. It seems a user-defined parameter/setting or change to the renko chart data set could be implemented to fix this. What is infinitely frustrating is that it is hard-wired into the system and as an admittedly small percentage of your and TS' customer base, pleas for a fix are universally dismissed.

But getting back to my question, if there is more to this backtesting problem than data deletion, could you please suggest who I could speak with to assess what I would be up against in formulating a work-around to facilitate valid backtesting?

Thanks.

[NinjaTrader_JoshP]

1. This is accurate behavior. When a bar closes and triggers a trade signal the next tradeable location is not the close of that trigger bar, but the open of the next bar. Whether there is a gap all of a sudden to cause a different open price or not is part of the reality of backtesting. There is no way to fill you at the close because that is already past data. To not "cheat" with fills in the past the next fill location possible is the open of the next bar.

2. I am not sure what you mean by data being deleted/ignored. Data is not deleted or ignored. If the data is within the session template you have set out for your strategy to use, all data within that session will be used. Any data outside of the session will be ignored. If you want data outside of the session to be taken into account please use a session template that contains those data bars. Please try using the "Default 24/7" session template.

[RJay]

billr,

If I were you I would try "WickedRenko" . Not only do they display wicks in real time, they are also template compatible with most chart types.

http://www.bigmiketrading.com/ninjatrader-programming/3834-wicked-renko-custom-bar-type-ninjatrader-7-a.html

Sadly, NT's version of Renko bars do not display all the price movement and you can't share templates with other chart types.

I don't know if this WickedRenko version will test any better. Haven't tested it, But I think it will.

Post your results for the rest of us.

Thanks,

RJay

[billr]

Thanks RJay. I will definitely check out WickedRenko. I was planning to use a programmer to enable a fix around this using 'wicks' as you describe.

I should have been more concise in my post. I wanted to acknowledge the data deletion problem and wanted to know if there are any other major known logic traps that are not as evident. I have found that it is sometimes difficult to get concise info on what known logic flaws exist in backtesting programs without having to wade through explanations of the risks and limitations of backtesting in general.

However, I have found Josh and the other support guys are very helpful, especially compared to eSignal and Tradestation.

Your suggestion demonstrates you understand what I was referring to.
Thanks again.

[billr]

Hello Josh,
Thanks for your response to my post regarding the renko backtesting logic. I respectfully offer the following comment:

RE: #1. The logic used in backtesting actually asserts a "cheat fill" in reverse. Instead of filling on data that was not there, the NT backtesting logic deletes actual trade data thereby forcing a fill at a point that would normally not be consistent with real market conditions. In the case of executions, this asserted "cheat" fill often occurs with a positive bias equal to one entire bar size thereby invalidating the test results.

RE: #2. The deleted data I am referring to that causes #1 and other problems is the common practice of designing the backtest logic to mimic the visual filtering feature of Renko. This is not a valid practice. Unlike time based charts, the Close and Open are created at the same time. Given RenkoBar[1] and RenkoBar[0], the Open of RenkoBar[0] can only occur at two price points.
One: Open[0] = Close[1] , or
Two: Open[0] = Open[1].
The bar does not open/close until price has reached a certain threshold in either direction. In the meantime, price data is being created as the price fluctuates. Any price data that falls outside the bar that is eventually formed is deleted from backtesting. This hard-wires a high percentage, false bias to the positive for executions, and other tick based functions such as stop loss, trailing stop and prof Targets. While the trading tools can be adapted, the executions cannot. This is the flaw I was trying to describe.

Thanks.

[RJay]

billr,

Both Renko and WickedRenko when building the current bar in realtime use a bar repainting program. It has to be this way because NT does not allow Open[0] to be redefined once the bar becomes active.

When the market reverses during a bar, NT saves the current bar data, then NT deletes the bar, then creates a new bar with saved data with a new Open[0] position.

I'm sure there is a good reason for doing it this way. doesn't mean I have to like it.

I was unable to program my way around this problem.

Back testing with historic data, bypasses this repaint function. That is why I recommended WickedRenko. It at least displays the full price movement for each bar.

RJay

[mrlogik]

Why not just plot the open where it actually is? This is one of the options I allow in my LogikRenko for NT6.5.

For example, a reversal bar open will plot at the prior bar close. Even though this isn't the way a Renko "is," it will solve the backtesting issue you are seeing.

just a thought.

[billr]

RJay,
I'm sure you are correct that there is a good reason for this from a programming perspective. And actually, I have no issue with the way the renko is formed live. It is only the backtesting that is a problem and the renko with all price data (wicked) should solve that.

Thanks again for your help.
Bill R.

[azuul]

1. This is accurate behavior. When a bar closes and triggers a trade signal the next tradeable location is not the close of that trigger bar, but the open of the next bar. Whether there is a gap all of a sudden to cause a different open price or not is part of the reality of backtesting. There is no way to fill you at the close because that is already past data. To not "cheat" with fills in the past the next fill location possible is the open of the next bar.
.

I am also not happy with the backtesting of Renko charts:

It only makes sense to get a fill at the !! CLOSE !! not at the open of the NEXT bar.

In Renko charts the close of the last bar is identical with the open of the next bar. So the fact is, NT is doing exactly what you stated is wrong: using a NON tradeable price - the closing of the past bar for execution (= opening of new bar = nontradeable because already past) which renders all results with RENKO useless and wrong.

The open of the next bar is normally already being traded and and "past", otherwise there would be no new bar. So in best case, only at very rare and few circumstances you would get a fill at this open of the next bar, normally it would be the closing price of the next bar in estimated at least 95% - 99.9% of all cases - or at least a tick or 2 higher than the OPEN of the next bar. But never ever the OPEN price.

One could assume slack and with a Renko of 2 add a slippage of one full future point to equalize, then adding another slippage for the real "slippage", but this solution is not very motivating.

Fact is: Results backtesting vs. live simulation with Renko chart has nothing in common, tick, sec, min, are at least similar or close.

Unfortunatelly backtesting with Renko is pointless in this Version without further modifications, please fix it!

[roonius]

I am also not happy with the backtesting of Renko charts:

The open of the next bar is normally already being traded and and "past", otherwise there would be no new bar. So in best case, only at very rare and few circumstances you would get a fill at this open of the next bar, normally it would be the closing price of the next bar in estimated at least 95% - 99.9% of all cases - or at least a tick or 2 higher than the OPEN of the next bar. But never ever the OPEN price.

You are wrong here. Renko does not have to exceed brick, only to reach it (it is not range bar).
So in liquid instruments most of the time open will actually be equal to previous close (it even can be lower) if bar is not reversing.
If bar is reversing, then it is different story - the open will be entire brick off (plus possible slippage)

Renko bars is visual tool and hard to use in strategies.

[azuul]

You are wrong here. Renko does not have to exceed brick, only to reach it (it is not range bar).
So in liquid instruments most of the time open will actually be equal to previous close (it even can be lower) if bar is not reversing.
If bar is reversing, then it is different story - the open will be entire brick off (plus possible slippage)

Renko bars is visual tool and hard to use in strategies.

Exactly my statement, the new open is the close of the last bar, but since it is already paid u will e.g. as a buyer hardly find any ASK for that price anymore, so you have to buy a tick higher. Reversal is a different story, but that can not be the assumption of the logic behind optimization (always and only considering reversal)

So what Renko optimization does: you always get the last paid price as exection price, without accepting that the open of the new bar is most of the times already past (otherwise there would be no new bar!) - Of course not true if price is moving in a small range up and down only, or if the instrument is super liquid. But just simulate the Fdax with brick size of 2 and u will see what I mean...

maybe something can be done, since backtesting vs. simulation is just too far apart, what is the point of offering Renko for backtesting ?