NT8 Intrabar Strategy back-testing pitfalls - Suggested Language Change Request

[tomg1970]

Good afternoon,

I just started developing a strategy in NT7 a couple weeks ago using a 15 minute bar chart and learned that NT7 back-testing does not support intrabar entries or exits. As I understand it, the only way around this is to develop the strategy using multiple time series, the second series using 100 ticks or something like that. Looking at NT8 I thought intrabar fills might be supported, especially since I saw screen shots from the optimization area that showed a section "Historical fill processing" with an option for "high" Order Fill Resolution, but when I use this option using 1 tick, it still appear the fill happens on the open of the next bar.

Now, I understand I can run my strategy as it is in Replay mode or Live, and it will work as I designed it. I can do as it was done in NT7 and use multiple bar time series, but I believe this would add much overhead to the strategy. Or am I missing some way of optimizing it?

Assuming it is what it is, I have a language suggestion for NT8. Would it be possible to add a hypothetical fill price to EnterLong, EnterShort, ExitLong, and ExitShort? The current method has the following overloads:
EnterLong()
EnterLong(string signalName)
EnterLong(int quantity)
EnterLong(int quantity, string signalName)

Could an optional HypotheticalPrice be added as so:
EnterLong([double HypotheticalPrice])
EnterLong(string signalName, [double HypotheticalPrice])
EnterLong(int quantity, [double HypotheticalPrice])
EnterLong(int quantity, string signalName, [double HypotheticalPrice])

The use of this parameter would cause NT to use the price as stated again the current bar rather than the next bar open, which must be between the current bars high and low, or and exception will be thrown. Also, if the parameter is used, then all back-testing reports will clearly state Hypothetical Price was used to prevent falsified reports. However, for my personal back-testing, I know it can be trusted (why would I lie to myself!). This parameter would be ignored when running live or in replay mode.

For example, if the current bar open at 45.00 and if the price rises to 45.20 I want to enter a long trade (at that point), and the bar will have a high of 45.50 and close at 45.35. In live mode the entry would happen at 45.20 or 45.21 with some slippage, but in back-testing it will show 45.35 by default, but if I use HypotheticalPrice like:
EnterLong(EntryPrice) where double EntryPrice = 45.20, then back-testing will use a fill price of 45.20 + back-test slippage amount as configured for the back-test. The same idea could be used for exits. This would allow me to not have to use multiple bar time series and allow me to back-test, optimize, and walk-forward test still only using 15 minute bars. Much less overhead to optimize a strategy with this implemented change. Any thoughts?

Thanks.. tom

[NinjaTrader_JessicaP]

Hello tomg1970, and thank you for your question.

With regard to NT8's intra-bar handling optimizations, provided all your intra-bar handling can be done in the section

[FONT=Courier New]protected override void OnMarketData(MarketDataEventArgs event)
{
    if (event.MarketDataType == MarketDataType.Last)
    {
        // handle here
    }
}[/FONT]

then you may handle these without creating an additional data series as you would need to in NT7 with the new Tick Replay feature. You can enable this feature by visiting Tools -> Options -> Market Data -> Show Tick Replay. The Tick Replay checkbox will then be available from your chart or strategy's data series window. This link has more information regarding developing for tick replay.

http://ninjatrader.com/support/helpGuides/nt8/en-us/developing_for__tick_replay.htm

With regard to the rest of your query, would it be possible for you to let us know if this feature request is adequately handled by NT8's new Market-if-Touched (MIT) order type? These are explained depth in this publicly available link

http://www.investopedia.com/terms/m/marketiftouched.asp

If your data feed provider does not support MIT orders directly, these are simulated by NinjaTrader.

[tomg1970]

Hi Jessica...

Thanks for your quick reply. So do I use "OnMarketData" to track price within a bar if I am correct where as OnUpdatebar during backtesting does not get called till the close of the bar? Should I then move Order Entry (and exits) to "OnMarketData" event then, since my fills are happening intrabar? Or can I just use this event to populate my current price variables, which can be referenced in OnUpdateBar code? Does OnUpdateBar only get fired when a bar closes or any time a bar changes (in back testing and live)?

I did not know about EnterLongMIT() order type. I am assuming this has to be called BEFORE a bar starts? Then if the bar touches that price intrabar, the order occurs and is recorded at that price. If I use this order type, do I still need to use "OnMarketData" event?

Can Price of EnterLongMIT be modified? Or must it be cancelled and re-issued?

Thanks again for your reply

[NinjaTrader_JessicaP]

Hi Jessica...

Thanks for your quick reply.

I am glad I was able to help.

So do I use "OnMarketData" to track price within a bar if I am correct where as OnUpdatebar during backtesting does not get called till the close of the bar?

Correct.

Should I then move Order Entry (and exits) to "OnMarketData" event then, since my fills are happening intrabar? Or can I just use this event to populate my current price variables, which can be referenced in OnUpdateBar code?

The latter is a better strategy.

Does OnBarUpdate only get fired when a bar closes or any time a bar changes (in back testing and live)?

This behavior is determined by the new three state Calculate property, documented here,

http://ninjatrader.com/support/helpGuides/nt8/en-us/calculate.htm

I did not know about EnterLongMIT() order type. I am assuming this has to be called BEFORE a bar starts? Then if the bar touches that price intrabar, the order occurs and is recorded at that price. If I use this order type, do I still need to use "OnMarketData" event? Can Price of EnterLongMIT be modified? Or must it be cancelled and re-issued?

MIT orders are essentially StopMarket orders that trade on the same side of the market as Limit orders, and opposite the market from Stop orders. Everything that is true about entering StopMarket orders - with the exception of the side of the market - is true about MIT orders. In order to access intra-bar data, you will still need to use Tick Replay with OnMarketData for MarketDataType.Last .

Please let us know if there are any other questions we may answer.

[tomg1970]

Good morning...

I tried as you suggested, but with little or no luck.

First, I configured NT8 to allow tick reply. Secondly, in my strategy, I moved my variables that I know will be updated intrabar to become global variables. Then I set the new bar values inside OnBarUpdate() as follows:
if (IsFirstTickOfBar == true)
{
currentOpen = Open[0];
currentHigh = Open[0]; // this will get updated via OnMarketData
currentLow = Open[0]; // this will get updated via OnMarketData
}

Then I updated them using:
protected override void OnMarketData(MarketDataEventArgs Event)
{
if (Event.MarketDataType == MarketDataType.Last)
{
currentHigh = Math.Max( Event.Price, currentHigh);
currentLow = Math.Min( Event.Price, currentLow);
currentPrice = Event.Price;
}
}

Then I tried to back-test and same results as before. The fills shows the price of the next bars open- as I have seen before. So I gave up on this approach as it will not back test accurately.

Next, I tried to use MIT order like so:
if (IsFirstTickOfBar == true)
{
if (currentLRS >= paramLinRegSlopeMin) // up slope
{
EnterLongMIT(orderQuantity, currentOpen + (calcBarRangeTicksEntry * TickSize), "VOL - Long_" + Instrument.MasterInstrument.Name);
Print("Long at: " + Time[0] + " Open: " + currentOpen + " Current: " + currentPrice + " Offset: " + (calcBarRangeTicksEntry * TickSize));
}
else if (currentLRS <= -paramLinRegSlopeMin ) // down slope
{
EnterShortMIT(orderQuantity, currentOpen - (calcBarRangeTicksEntry * TickSize),"VOL - Short_" + Instrument.MasterInstrument.Name);
Print("Short at: " + Time[0] + " Open: " + currentOpen + " Current: " + currentPrice + " Offset: " - (calcBarRangeTicksEntry * TickSize));
}
}

I checked to use MIT check-box on my back-test and the back tested and the results looked great, but I then noticed the fill prices was the price of the current bar open and NOT the bar up + 20 ticks as it shows in the output of my print statements. This price was far from available when the trade actually took place.

On the later example, I also tried to change order handling for my back-test to "High" resolution and whether I choose 1 minute or 1 tick, the back test yields no trades, but my Print statements still show I should have had trades! I am confused and frustrated...

Is there any simple example strategies that show intrabar trading that works in back-test?

Not sure what to do next. :-(

Appreciate any more suggestions. A sample strategy that works in terms of back-testing accuracy would be great.

Thanks.. tom

[NinjaTrader_JessicaP]

As far as Tick Replay samples are concerned, while NT8 does not distribute with sample tick replay strategies, I would like to recommend reviewing the built-in VolumeProfile indicator.

Based on this indicator, I have made an educational sample which only places trades in markets with a minimum liquidity, attached to this reply.