Incorrect Fill Price even when using tickreplay and having bid/ask.

[NinjaTrader_Brett]

Although yes you can do what DavE suggested you must be careful since your working around how its intended to work and coding in 'look ahead' logic into your code. You obviously can't look ahead during realtime so I would not advise adding that code unless you understand what your getting into there in terms of having separate backtest logic vs realtime logic.

As to utilizing the bid/ask which you re-clarified is what your after this is something we have our list to consider looking into but not something we do currently or anytime in the near future.

-Brett

[Trader212310]

Hi Ray,

Is DaveE correct in that what I mentioned in post #30 is currently possible?

regards

[DaveE]

Hello Trader212310

You don't need a new feature in NT to get what you are requesting in post #30.
All you need to do is to update your signal to use Bars.GetOpen(CurrentBar+1) then your signal will occur one bar earlier, and since the Fill Engine is also using Bars.GetOpen(CurrentBar+1) you will get the fill at the price you are expecting, at the time you are expecting.
Proviso: I have not actually tried this on a tick data series, I just know it works on minute based series.

[koganam]

Hi Ray,

Apologies for not being more clear on what I would be using the functionality in post #27 for.

I understand there are ways to custom code an NT8 strategy so that it "sends" orders to 1 tick Bid bars and 1 tick Ask bars accordingly (and fills them at the according Bid or Ask Price) when used in the strategy analyzer
With that in mind, lets assume the following:


1. I have imported all Bid ticks and Ask ticks (that happened both on and in between trades) into NT8.
2. The numbers 100 and 106 mentioned in my previous post were Ask prices, not Last prices
3. I have programed my strategy to send Long entry orders to the 1 tick Ask bars.

The functionality requested in post #27 would be so that my Long entry would fill at 100 instead of 106.

Hope this clarifies why I am looking for this feature.

And now that you state that it is your own lack of clarity that caused me to respond with a response to the question as posed, and not as in your head, I expect you to send me another PM apologizing for the rude one that you sent me yesterday threatening to have NT revoke my access to the forum. As if you even could.

If I do not get that forthwith, I shall do what I have never done, and publish in this thread said PM that you sent me, rudely giving me orders as if I were your personal slave.

I am reasonably sure that that is where we shall end up as people of your ilk, unlike me, are unable to admit that you made a mistake. I have met you many times under your different names.

[Trader212310]

Hi Ray,

Apologies for not being more clear on what I would be using the functionality in post #27 for.

I understand there are ways to custom code an NT8 strategy so that it "sends" orders to 1 tick Bid bars and 1 tick Ask bars accordingly (and fills them at the according Bid or Ask Price) when used in the strategy analyzer
With that in mind, lets assume the following:


1. I have imported all Bid ticks and Ask ticks (that happened both on and in between trades) into NT8.
2. The numbers 100 and 106 mentioned in my previous post were Ask prices, not Last prices
3. I have programed my strategy to send Long entry orders to the 1 tick Ask bars.

The functionality requested in post #27 would be so that my Long entry would fill at 100 instead of 106.

Hope this clarifies why I am looking for this feature.

[DaveE]

...
Lets assume I got an entry signal at the 2pm close of a one hour bar and the closing price of this 2pm one hour bar was 100. Lets assume (its a thinly traded symbol) and the next tick does not come until 2:18 pm and that tick's price in 106.

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.
...

Trader212310, When you say

I got an entry signal at the 2pm close of a one hour bar

You seem to be assuming that in OnBarUpdate(), where I assume your 'signals' are generated, is called at the logical time of close of the 2pm bar, ie at the same time as is shown in Time[0], but it is not.
Assuming there is no session break starting at 2pm (if there was see my post #21), then this is what would happen in Realtime (assume 60 minute Ask data series so the prices in your example are Ask):
At 2:18pm OnBarUpdate() is called. CurrentBar points to the bar with Close[0] = 100 and Time[0] gives the logical Close time for that bar (2pm). But if you Print(DateTime.Now) you will see the actual time is 2:18pm. If you Print(GetCurrentAsk()) this will give 106.
If you Print(Bars.GetOpen(CurrentBar+1) this also gives you 106. This is correct since its the price that a Market Buy will likely get filled at (if broker has enough quantity). Lets say in this OnBarUpdate() your code indicates a signal to Buy at Market and issues EnterLong(), then it will most likely get filled at 106 (not 100). Note that your signal is not being triggered at 2pm but at 2:18pm, so your requirement 1 in post #17

Fill orders at whatever the price was at the moment the order signal was generated

is happening Realtime (and also correctly simulated in Historical see below).

Ideally A Historical Fill Engine (used in both Strategy Analyzer and for processing live strategies while they are in State.Historical) should be simulating what would happen Realtime as closely as possible:
Here is what happens with the same example running in State.Historical:
When OnBarUpdate() is run for Time[0]=2pm: If you Print(GetCurrentAsk()) you get 100. This seems to support Trader212310's argument, but I think it is a mistake (see here).
If you If you Print(Bars.GetOpen(CurrentBar+1)) you get 106, which is the correct value simulating what would have been a Market Buy fill price in Realtime.
If your code does exactly the same thing regardless of Realtime or Historical then your 'signal' should do the same thing as in Realtime (it might not if the logic includes GetCurrentAsk). Lets say your code indicates a signal to Buy at Market and issues EnterLong(), then the Fill Engine will fill it at 106 correctly simulating what would have happened Realtime.
You need to think of your 'signal' code in OnBarUpdate() in Historical as working at the the open of a new bar since that is what happens Realtime.
This means that if your 'signal' is working off Close[0] then your signal is out of date. You could try driving your signal off Bars.GetOpen(CurrentBar+1) instead which is the latest price at OnBarUpdate().
Even NT development sometimes make the mistake of thinking that Close[0] is the latest price. See for example the incorrect rule for Historical Stop price

[NinjaTrader_Ray]

Hi Ray,

You need to make the fill price be 100 in this situation. Not 106.

Again, your platform is fantastic and I don't mean to sound harsh, but not doing this is just as I mentioned in post #16.

Thanks.

I am sorry but I disagree 100%. To try and illustrate my point.

1. Bar closes with a price of 100
2. Your strategy now executes logic and it decides to enter a market order
3. The next price comes in is where your market order is filled

It is chronologically impossible to fill the order in step 2 with a price seen in step 1 from the past. I don't think there is any backtesting tool that would do what you want.

[Trader212310]

Hi Ray,

You need to make the fill price be 100 in this situation. Not 106.

Again, your platform is fantastic and I don't mean to sound harsh, but not doing this is just

downright irresponsible and reckless

as I mentioned in post #16.

Thanks.

[NinjaTrader_Ray]

Hi Ray,

Would the fill price be 100?

No, the fill price would be 106. On the close of the bar when you issue your market order, the price of 100 is now in the past. 106 is the next price after the market order is issued hence its filled at that price.

[Trader212310]

Hi Ray,

Would the fill price be 100?

[NinjaTrader_Ray]

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.

Exactly what I am saying and what is expected. On the tick of 100 which is the last seen price for the bar, your strategy now will process your trade logic and issue an order (market order for the sake of this discussion), the next incoming tick is 106 which is the open of the next bar which is now where your order is filled.

[koganam]

The logic you are saying would cause the strategy analyzer fill the order at price 106.

If you use a market order.

I would need it to fill the order at the 100 price.

Which it will if you use a Limit Order and the new bar has a low lower than 100. In other words, the entry bar will have to cut the entry price, which is what is most likely to happen even a live market.

[Trader212310]

Hi Ray

I don't think your

That is exactly what is happening.

quote in post 20 is correct.

Lets assume I got an entry signal at the 2pm close of a one hour bar and the closing price of this 2pm one hour bar was 100. Lets assume (its a thinly traded symbol) and the next tick does not come until 2:18 pm and that tick's price in 106.

The logic you are saying would cause the strategy analyzer fill the order at price 106.
I would need it to fill the order at the 100 price.


P.S.

FYI:
Regarding my post 18 quote below

Why not have it fill at whatever the price is at the nanosecond after the bar closes?

In the scenario I've described in this post (post 22) 100 (not 106) is what the price was the nanosecond after the 2pm one hour bar closed.

[DaveE]

Why not have it fill at whatever the price is at the the nanosecond after the bar closes?

This would not make sense if the bar was logically closed due to a session break.
(Ray's answer seems to assume there is no session break).
For a simple example, and avoiding the bid ask spread issue, lets say you are trading on a 30 minute Ask data series, in ET time zone using Trading Hours=Nymex Energy RTH (sessions 9am to 2:30pm):
In Realtime you might think of the bar ending at 2:30pm as closed if you are a nanosecond after 2:30pm on a Friday, however NT does not actually close it until it gets the first Ask tick after new session opens at 9:00am the next Monday. Only at that point does OnBarUpdate() fire.
Its possible that NT receives some Ask ticks during the defined session break, and these might get stored in Historical Data, but it would not make sense to fire OnBarUpdate() for these since the Broker would presumably reject them (no order connection).
So in Historical it is correct for the Fill Engine to wait until the end of a session break (if there is one). That means it is correct for market buy orders on the Ask data series to fill at GetOpen(CurrentBar+1). If instead it tried to fill at some rogue Ask tick that arrived during the session break it would not be simulating what happens in live.

Incidentally:

I think the fact that GetCurrentAsk(), when used in Historical, returns Close[0] is partly to blame for the confusion, since it helps to perpetuate the myth that OnBarUpdate() is called at the start of a session break. Because it is actually called at the end of a session break it would make more sense for GetCurrentAsk() to return GetOpen(CurrentBar+1) see here

[NinjaTrader_Ray]

Why not have it fill at whatever the price is at the the nanosecond after the bar closes?

That is exactly what is happening. The very next tick after the bar closes is the tick that opens the next bar.

Using bid/ask data is another discussion and an item we have already put on our suggestion list.