Premature Stop Outs in Backtesting

[MrOrange]

Hi There,

I've got my strategy entering properly now, i think. Big accomplishment!

I'm having a problem in the strategy analyzer, reporting a LOT of my trades as stop outs even though they're not.

It's a little hard to explain so i'll attach some screenshots, but basically if i have a long entry via a buy stop order at a particular price and it's picked up on a bullish candle, and that bullish candle is quite large, the strategy analyzer reports the trade as a stop out, even though it's not, as if the very large bullish candle picked up my trade, then completely collapsed, hitting my stop, then went all the way back up to being a big bullish candle.

What's the best way to work around this problem? Is there a way to get the strategy analyzer to actually look inside each bar to see what happened? Or is there an easy way to delay implementing the stop loss until the bar the trade is entered on, is closed?

Thanks!!

[MrOrange]

Interestingly, if i put this in my code, in the OnBarUpdate section...

if(BarsSinceEntry(0, "myLong", 0) > 1)
{
SetStopLoss(150);
SetProfitTarget(150);
}

so that my stop and target wouldn't be 'active' until 1 bar after the entry. It didn't work, i still get falsely stopped out on the entry bar.

then i tried this:

if(BarsSinceEntry(0, "myLong", 0) > 20)
{
SetStopLoss(150);
SetProfitTarget(150);
}

same thing! it wasn't waiting 20 bars, it stopped me out on the entry bar, again.

however, if I do THIS, the problem goes away (though, stops are not activated for 200 bars, so it's not a solution, but it's hard to imagine that BarsSinceEntry actually works properly given this behavior)

if(BarsSinceEntry(0, "myLong", 0) > 200)
{
SetStopLoss(150);
SetProfitTarget(150);
}

That makes no sense to me...

[MrOrange]

One last update. I've now tried using multiple timeframes to solve this problem with no luck, following the "SampleMultiTimeFrame.zip" provided elsewhere on these forums.

Basically in Initialize() i added:

Add(PeriodType.Tick, 300); //my primary data series
Add(PeriodType.Tick, 30); //my secondary data series

and then in OnBarUpdate() added:

if (BarsInProgress == 2)
{
SetStopLoss(150);
SetProfitTarget(150);
}

and then wrapped all the other logic in my strategy (ie, the code to determine the entry), as in this, within the OnBarUpdate() section.

if(BarsInProgress==1)
{
//the rest of my code
}

Exact same symptoms as in my first two screenshots. Any help would be greatly appreciated!

Thanks!

[NinjaTrader_AdamP]

MrOrange,

If you notice, in the second screen shot the low of the candlestick is indeed hitting your stop loss. A bar can move a lot before it closes positive, including periods where it is a bear bar. Its possible your entrance is triggering somewhere in the middle, then the price goes down enough to trigger the stop before the price again moves in the positive direction. I would recommend using Market Replay with your strategy active to get a better idea how the trades are being opened and managed inter-bar.

For more information on Market Replay, please see the following link.

http://www.ninjatrader.com/support/helpGuides/nt7/index.html?market_replay.htm

Please let me know if I can assist you further.

[MrOrange]

Hey Adam,

Thanks for the response. That's not what's happening unfortunately. You can see the entry is right near the top of these candles, not in the middle. The candles do not 'collapse' and hit the stop after the trades are picked up, only to rebound and close very bullish. It wouldn't be happening on EVERY large bullish candle that picks up one of my trades, if that were the case. It would only happen on some of them, right? Also, I can open up a 10 tick chart, examine any of these false stop outs, and see that the trade should DEFINITELY not be stopped out, even though the strategy is reporting it.

Any other ideas?

Cheers

[koganam]

Hey Adam,

Thanks for the response. That's not what's happening unfortunately. You can see the entry is right near the top of these candles, not in the middle. The candles do not 'collapse' and hit the stop after the trades are picked up, only to rebound and close very bullish. It wouldn't be happening on EVERY large bullish candle that picks up one of my trades, if that were the case. It would only happen on some of them, right? Also, I can open up a 10 tick chart, examine any of these false stop outs, and see that the trade should DEFINITELY not be stopped out, even though the strategy is reporting it.

Any other ideas?

Cheers

I seem to remember that NT assumes the worst in the backtest situations; that if a bar touches both the profit target and stop loss, then the stop loss was hit first. That also then means that any bar that touches the stop loss is assumed to be stopped out. Just one of the assumptions that was chosen to be made, that really cannot be helped in the face of a lack of granularity.

[MrOrange]

Hey Koganam,

Thanks for the response. That seems to be correct, ninja is assuming that i'm getting stopped out on the entry bar, because it's low was below my stop. Lack of granularity as you said. However, there should be two possible solutions for this. 1.) Don't implement your stop until one price bar has formed after your entry (using the BarsSinceEntry function) or 2.) Use a second, smaller timeframe, and implement the stops and targets on that time frame (while still using the larger timeframe to determine entry). However, neither seems to be working for me.

Cheers.

[MrOrange]

Hey There,

I was just hoping to bump this thread to get some attention from Ninja Support. What's the best way for me to avoid these false stop outs, and can you point me in the right direction to accomplish it?

Thanks!

[koganam]

Hey Koganam,

Thanks for the response. That seems to be correct, ninja is assuming that i'm getting stopped out on the entry bar, because it's low was below my stop. Lack of granularity as you said. However, there should be two possible solutions for this. 1.) Don't implement your stop until one price bar has formed after your entry (using the BarsSinceEntry function) or 2.) Use a second, smaller timeframe, and implement the stops and targets on that time frame (while still using the larger timeframe to determine entry). However, neither seems to be working for me.

Cheers.

Option2 should probably be OK. However, I think that you are going to have to enable plotting of the finer BarSeries, to be sure exactly what is happening. Have you properly isolated the processing of your BarSeries?

[MrOrange]

Thanks again Koganam,

I'm not too sure what you mean. When i'm running a backtest against two data series, do i have to plot both in the strategy analyzer? It seems to only allow me to plot one.

Certainly i can examine any particular trade on a smaller timeframe, manually, and see if it was a legit stop out or not (95% aren't).

In terms of isolated the processing of my bar series, again, no idea what you mean. How do i isolate processes? Here's what my code for it looks like, which i've taken from the SampleMultipleTimeframe examples found elsewhere (but even with this code, the problem still remains).


Basically in Initialize() i have:

Add(PeriodType.Tick, 300); //my primary data series
Add(PeriodType.Tick, 30); //my secondary data series

and then in OnBarUpdate() i put:

if (BarsInProgress == 2)
{
SetStopLoss(150);
SetProfitTarget(150);
}

and then wrapped all the other logic in my strategy (ie, the code to determine the entry), as in this, within the OnBarUpdate() section.

if(BarsInProgress==1)
{
//the rest of my code (which determines trade entries)
}

[NinjaTrader_RyanM1]

Hi MrOrange,

This is expected for backtesting with set statements. The set statements will look at all available bar objects and cannot be submitted to a specific series only. If the stop can be filled on the entry bar, then it will be filled on the entry bar.

If your goal is create a real time strategy, you can test this operation in market replay and the set statement will evaluate tick by tick.

If your goal is to simulate this as best you can in a backtesting environment, your strategy will need to be changed a good bit and require working with advanced handlers:

• You need to use ExitLongStop() instead of SetStopLoss(). This allows submission of the order to the smaller series using the advanced overload.
• ExitLongStop() does not submit automatically upon entry execution like set statements. You can follow this sample to have the same timing of order submission.
• You will have to set liveUntilCancelled = true, which means you may need to manually cancel the order and manage its IOrder values. This sample can help with manual CancelOrder()

[MrOrange]

Hi Ryan,

Thanks a lot for your reply. Just to be 100% certain before i start digging into this, you are saying i'll need to use ExitLongStop() and manage IOrder values along with a second, smaller, timeframe?

Thanks!

[NinjaTrader_RyanM1]

Yes, you still need to add a smaller time frame to get the granularity, and only ExitLongStop() offers capability to submit to this secondary series. Set statements will look at all available bar objects to determine a fill.

[koganam]

Yes, you still need to add a smaller time frame to get the granularity, and only ExitLongStop() offers capability to submit to this secondary series. Set statements will look at all available bar objects to determine a fill.

Just to keep it straight in our minds again, when using managed orders, it is not possible to create OCO orders other than using Set...() statements, or coding a custom functions based on OnOrderUpdate() and/or OnExecution(). Is that right?

[NinjaTrader_RyanM1]

In a managed environment, there's no explicit OCO ids except for orders placed with set statements. A lot of the cancellation is handled by the strategy though. If a position associated with a working exit order is closed, then the exit order will be cancelled by the strategy.