Bid/Ask Volume Data for High Order Fill Resolution on Tick Level

[eicke]

Hey folks,

In general what I'm aiming for is the most accurate backtest for a strategy that:

1. Uses Heiken Ashi on certain minute basis
2. Sets a stop/limit order after the Heiken Ashi bar was completed
3. Needs ask/bid volume of the aforementioned bar as a signal for step 2.
4. Uses take profit and stop loss on tick level, hence needs the highest accuracy possible

I started with tick replay and accumulated the ask/bid volume via

protected override void OnMarketData(MarketDataEventArgs e) {
  if(e.MarketDataType == MarketDataType.Last) {
    if(e.Price >= e.Ask){
      buys += e.Volume;
    }else if (e.Price <= e.Bid){
      sells += e.Volume;
    }
  }
}

And then realized that the Standard Order Fill Resolution does not produce very great results. I simply wanted to change it to High but then stumpled upon this:

[...] Furthermore you cannot combine both Tick Replay and High Order Fill resolution.

Now when disabling tick replay and opting for High Order Fill Resolution I have the issue that my previously used method for accumulating bid/ask volume does not work anymore.
Q1: How else would I get this data?

Q2: And is my assumption correct that with those mentioned strategy parameters Calculate.OnBarClose combined with High Tick Level Order Fill Resolution should be sufficient since I only need minute bar granularity for my signal for the limit/stop order but the high tick level for the order fill resolution to achieve most accurate results?

Thanks!

PS: I use Rithmic as connectivity provider which according to this should give me my desired data.

[NinjaTrader_Jesse]

Hello eicke,

If you need to use high order fill resolution and tick replay then you would need to add a secondary series. You can see an example of submitting orders to a more granular series here: https://ninjatrader.com/support/help...ipt_strate.htm

You would then just use the standard selection for the fill processing as you are re creating that on your own by adding the secondary series.

For your second question. you could run your script on a minute series using OnBarClose to control your logic.

The TickReplay would be needed for OnMarketData to be called historically, OnMarketData will otherwise work in realtime without tick replay, you would just use the passed in market data events and the Price property: https://ninjatrader.com/support/help...htsub=onmarket

For more granular fills you can add a secondary series like a 1 tick series to submit orders to.

Those items would allow the script to have the most accurate results in terms of fill processing and allow you to do the items you mentioned.

[eicke]

Hey Jesse,

thanks for the answer, it already helped.
I added a second data series AddDataSeries(Data.BarsPeriodType.Tick, 1) and also adapted my orders to be placed on the added series by using setting barsInProgressIndex to 1 and using another "advanced order handling" method. I had to set isLiveUntilCancelled to true because I wanted it to survive longer than one tick and added some additional cancel logic.
For far so good, while the results of the backtest changed - so something is working differently - the resulting trades look wrong to me.
I have set SetProfitTarget(CalculationMode.Ticks, 4, isMIT=false) & SetStopLoss(CalculationMode.Ticks, 4) and get a results ranging for TP (4-16 ticks; avg. 5.6 ticks in spite of no MIT) and SL (4-33 ticks; avg 8 ticks) for an instrument like ES which is not that volatile and as far as I understand slippage is not artificially culculated and added.

Do you have any explaination for these results and could maybe point me in any helping direction?

Thanks!

[NinjaTrader_Jesse]

Hello eicke,

I couldn't really say what may have changed there from the given details, are you certain that it is slippage and not when the target is being submitted which is causing the difference?

[eicke]

Hey Jesse,
I ran it again with MES and exported the orders to Excel to ease filtering and sorting by adding an additional column for the resulting difference.
For the stop loss market orders I would expect some slippage in real life, the numbers are a bit too crazy though.
Furthermore, especially for the take profits which are limit orders I cannot understand the behavior/results.
I additionally attached the applied backtest parameters.

Do you have any explaination for these results?

Thanks!

[NinjaTrader_Jesse]

Hello eicke,

From the given results I would not be sure, do you see similar results for market orders when using the SampleMACrossOver? Do you see this only when using tick replay/added granularity?

[eicke]

Hey NinjaTrader_Jesse ,

I now took the simple sample code from your previously suggested SampleIntrabarBacktest_NT8 project and adapted it slightly with the additional stop loss and take profit - see picture below.

When running a super simple backtest with normal 5 mins candles as basis with or without tickreplay I get a quite a lot of outliers in spite of using the 1 tick series I submit orders to and the isMIT false for the take profit.
Does this behavior seem right to you? Also happy to perform further tests for you!

Lastly, when removing the second data series and changing it the order fill strategy to high on tick level I get exactly what I expect 4 ticks fills. However, then OnMarketData() does not provide me with the ask/bid data which I need.


Thanks!

[NinjaTrader_Jesse]

Hello eicke,

When using both TickReplay and the secondary series for fills you are essentially running into the same issue that the analyzer tries to prevent by not allowing you to use high fill resolution with tick replay. While you can get around that by adding a secondary series to allow more granular fills you will also see that the fill engine may have unexpected difficulties processing that situation.

Using TickReplay with a secondary series would mainly be useful for allowing a strategy to work historically before entering realtime when the requirement is that OnMarketData is needed for a historical calculation that may otherwise take a long time to start if only used in realtime.

One note from the documentation on TickReplay that may also be relevant here is that it is not really intended for accurate bid ask volume:

Tick Replay ONLY replays the Last market data event, and only stores the best inside bid/ask price at the time of the last trade event. You can think of this as the equivalent of the bid/ask price at the time a trade was reported. As such, historical bid/ask market data events (i..e, bid/ask volume) DO NOT work with Tick Replay. To obtain those values, you need to use a historical bid/ask series separately from Tick Replay through OnBarUpdate()

https://ninjatrader.com/support/helpGuides/nt8/developing_for__tick_replay.htm?zoom_highlightsub=tickreplay

If the goal is to accumulate the bid and ask volume historically then adding historical 1 tick bid/ask series would be better suited for that task if your provider offers historical bid/ask data. Going that way would allow tick replay to be turned off and you would still use the other secondary series for granular fill processing.

[eicke]

Hey NinjaTrader_Jesse,

I'm still a bit confused whether I can achive what I want with NinjaTrader or not.

To repeat, the objective needs order processing (fills for limit orders as well as SL+TP) on tick level. Additionally I need ask/bid volume on minute level - which I could also somehow manually accumulate (I can code).

As far as I understood and from my tests, I only get order processing on tick level by using High Order Fill Resultion. But for bid/ask volume the normal ways of using either Tick Replay (for bid/ask data from OnMarketData()) or by adding additional data series which provide me with MarketDataType.Bid to then accumulate the volume on this, do not work with High Order Fill Resultion.

You then suggested adding an additional data series with tick level accuracy which would allow me placing my order on this series/tick level, however, result have shown me not the desired accuracy for e.g. take profit or stop loss.

This understanding and fact basis lead me to beliebing that there is no available option to achive my objective with NinjaTrader.
Would you agree or do you see any mistake in the resoning or test results? If so I'd kindly ask you to elaborate again.

Thanks!

[NinjaTrader_Jim]

Hello eicke,

Adding intrabar granularity for order fills is separate from adding Tick Replay which allows for intra bar strategy logic with historical processing.

Set methods will use the first bar context of an instrument for the fill series, so in this case, the primary data series would be the fill series for the Set method.

If you are adding a single tick data series to the strategy, it would be recommended to modify the logic to use Exit methods (Note: Exit methods do not use order routing server based OCO) or to use Unmanaged Approach in the OnExecutionUpdate method to deploy the target/stop where you can direct the target and stop to the single tick data series.

The attached strategy may be compared with the original SampleOnOrderUpdate strategy. This can show how to use Exit methods in OnExecutionUpdate instead of Set methods, and to direct the orders to the single tick data series.

Using OnOrderUpdate and OnExecutionUpdate - https://ninjatrader.com/support/help...and_onexec.htm

The above would be the programmatic changes to a strategy to support Tick Replay and intra bar order fills.

Alternatively, the single tick data series does not have to be added and Tick Replay may not be needed: Market Replay data in the Playback Connection could be used, and the Control Center's New > Trade Performance window and Strategies tab > Right Click on Strategy > Realtime Strategy Performance Report could be used to track the changes.

This will result in the strategy processing logic following OnEachTick/OnPriceChange Calculate modes, with orders filled with intrabar data.

[eicke]

Hey Jim,

thanks for the idea of setting the stop/target on OnExecutionUpdate() - it works as desired!