Market Replay Filled Order At Ask Price That Did Not Exist

[Matheyas5]

Thanks, could I fix this issue by making a custom fill algorithm, as explained in this link?

https://ninjatrader.com/support/help...fill_types.htm

Or will the issue come up even if i try to correct it with a custom fill algorithm?

[NinjaTrader_JessicaP]

The short answer to your question is no. With a custom fill strategy, you would still be limited to the data coming into your strategy, and as we will be able to see, the data you need is not available.

To answer this question, I added the following line of trace code :

[FONT=Century Gothic][FONT=Courier New] Log(String.Format("Current Bid: {0}\r\nCurrent ask: {1}\r\nCurrent Close: {2}\r\n", GetCurrentBid(), GetCurrentAsk(), Close[0]), LogLevel.Information);[/FONT]
[/FONT]

This produced this in my log :

4/21/2016 10:07:23 AM|1|4|Current Bid: 999.5
Current ask: 999.5
Current Close: 999.5

4/21/2016 10:07:23 AM|1|4|Current Bid: 1000.5
Current ask: 1000.5
Current Close: 1000.5

4/21/2016 10:07:23 AM|1|4|Placing a trade at ask price
4/21/2016 10:07:23 AM|1|4|Placing a trade at bid price
4/21/2016 10:07:23 AM|1|4|Placing trade at last price

I did notice that when I imported from a text file, that I had generated the following minute data :

Ask

Date Time Open High Low Close Volume
4/15/2016 12:01 PM 4/15/2016 12:01 PM 1000 1000 1000 1000 46

Bid

Date Time Open High Low Close Volume
4/15/2016 1:01 PM 4/15/2016 1:01 PM 999.5 1000.5 999.5 1000.5 26
4/15/2016 1:00 PM 4/15/2016 1:00 PM 999.5 999.5 999.5 999.5 11

Last

Date Time Open High Low Close Volume
4/15/2016 2:01 PM 4/15/2016 2:01 PM 1000 1000.5 1000 1000.5 524
4/15/2016 2:00 PM 4/15/2016 2:00 PM 999.5 999.5 999.5 999.5 32

I therefore think it is safe to conclude that a custom fill strategy won't work. This is because even though the Ask price definitely never reached 1000.5, being exactly 1000 in both the Tick and Minute data, we still received a log message for a trade being placed at 1000.5 .

I also believe it is safe to conclude that, when backtesting a strategy, the Last traded price is used as the Bid and Ask prices, and the actual Bid and Ask data is simply discarded.

[Matheyas5]

Hi Jessica

I also believe it is safe to conclude that, when backtesting a strategy, the Last traded price is used as the Bid and Ask prices, and the actual Bid and Ask data is simply discarded.

So your saying the following is true when using the Strategy Analyzer to Backtest and/or Optimize a strategy that contains multiple data series?

1. Its impossible for Short Entry orders to fill at the Bid Price even if the strategy is explicitly coded to send its Short Entry orders to a 1 Tick Bid data series?
2. Its impossible for Short Exit orders to fill at the Ask price even if the strategy is explicitly coded to send its Short Exit orders to a 1 Tick Ask data series?


If this is what your saying, please be aware that this means its absolutely impossible to get accurate backtest results in Ninjatrader's strategy analyzer no matter how one codes their strategy

I don't see myself using Ninjatrader anymore if this is the case (which is a shame for ninjatrader because my high frequency strategies bring a large amount of trade volume/commisons to your brokerage).

Before I quit Ninjatrader, I'd really appreciate if you had a senior staff member review this thread (ideally NinjaTrader_Brett) to verify everything i've been told is accurate.

Its truly a shame Ninjatrader doesn't take accurate backtesting more seriously, its such a great platform otherwise.

[NinjaTrader_ChelseaB]

Hello Matheyas5,

From the help guide on GetCurrentAsk / GetCurrentBid:
"Note: When accessed during a historical backtest, the close price of the evaluated bar is substituted."
http://ninjatrader.com/support/helpG...currentask.htm
http://ninjatrader.com/support/helpG...currentbid.htm

Also, in the help guide for OnMarketData / OnMarketDepth:
"This method is not called on historical data (backtest)"
http://ninjatrader.com/support/helpG...marketdata.htm
http://ninjatrader.com/support/helpG...arketdepth.htm

If you want bid and ask historically you must add these as a secondary data series using the Add() call with the correct MarketDataType. Then properly style your script to call the correct BarsInProgress.

Add(string instrumentName, PeriodType periodType, int period, MarketDataType marketDataType)

http://ninjatrader.com/support/helpGuides/nt7/add3.htm

[NinjaTrader_ChelseaB]

Hello Matheyas5,

I'm looking further into your first post.

From my understanding the ExitShortLimit placed on BarsInProgress 2 (the ask series), filled at a price that was not available from the ask ticks, is this correct?

Was the order placed on the correct side of the market?

I've made a quick script to test placing limit orders on both sides. I've also made a video where I am looking for fills that are incorrect.
http://screencast.com/t/YA9STTU0sxHf

In the video I am showing historical data, however, I am currently testing with Market Replay.

I would like you to set up your chart the same as I have in the video so that we can see that the orders are filling on the incorrect side of the market when there is a bid or ask tick.

[Matheyas5]

Hi Chelsea,

Post #1 of this thread has a strategy attached to it called ReplayExample.zip.
Post #1 of this thread also explains in detail an issue that came up when I ran ReplayExample.zip in Market Replay.

Post #7 of this thread explains in detail an issue that came up when I ran ReplayExample.zip in the Strategy Analyzer.

The issues I brought up in post 7# and post #1 are the only issues i'm trying to solve.
I'd recommend ignoring all other posts in this thread aside from post #1 and post #7. I figure this will make things less confusing on your end.



Thanks for your help.

P.S.

Before I test the script you included in post #19, I'd appreciate if you first used my ReplayExample.zip to do the exact same tests I did in Post #7 and Post 1# (using the exact same data I did) to see if you get the same exact issues I did.

-Post #7 includes a link to the .csv files of historical data I used for post #7's test.
-The link below contains the two Market Replay files I used to do Post #1's test.

https://www.dropbox.com/sh/oo95l3dy4tb1nya/AAAT1EyUGCgk5iQ6T6hQDccja?dl=0

All timestamps in these files and posts are EST.

[Matheyas5]

Hi Chelsea,

I just wanted to give you a heads up that I added the section quoted below to post #21 a few hours after i originally posted post #21. I figured I'd tell you here just to make sure you noticed it.

P.S.

Before I test the script you included in post #19, I'd appreciate if you first used my ReplayExample.zip to do the exact same tests I did in Post #7 and Post 1# (using the exact same data I did) to see if you get the same exact issues I did.

-Post #7 includes a link to the .csv files of historical data I used for post #7's test.
-The link below contains the two Market Replay files I used to do Post #1's test.

https://www.dropbox.com/sh/oo95l3dy4...T6hQDccja?dl=0

All timestamps in these files and posts are EST.

Thanks and have a great weekend.

[NinjaTrader_ChelseaB]

Hi Matheyas5,

In post #1 you mention the following:

Anyway, you'll see in 2.jpg that the short exit called "TakeProfitExit" was filled at 1884.00, but no Ask bars on the chart ever dropped to 1884.00.

How do you know this?
The script you have made is hard to follow. The entry is placed when a MACD Avg using 1 tick data crosses the MACD Macd with 1 tick data.
Since its unlikely that the avg and macd prices are going to cross each with 1 tick as the input series, its very difficult for this strategy to place trades.

I was attempting to make an easy test where the entry is placed immediately and often so that we can see what is going on.
I want to assist you and to do so, I need to know what the issue is. With the screenshot and the script you have posted there is no evidence that an order has filled on the wrong side of the market. There is no output go along with this to see where the order is placed and where the order filled and what the ask and bid data was at the time.

I have created a script that attempts to test the behavior you are describing so that I do not have spend time trying to get your strategy to work.

In my post I have included a video showing the bid and the ask data so that you can see where the ask price is for buy orders and whether the order is on the wrong side of the market. (For buy limit orders we only need the ask price or better)

I am attempting to clarify with you what the actual behavior you are experiencing is, and how you have gotten to that conclusion.

My test script was to test both the entry and exit side of things and be able to show the prices where the order is being submitted and where it is being filled.

At this time, I do not have enough information from you to see that there is an issue.

However, as you have requested, below is a video of me testing your script.
http://screencast.com/t/qG02H8tBdqX

[Matheyas5]

Hi Chelsea,

In regards to this question

In post #1 you mention the following:


Code:
Anyway, you'll see in 2.jpg that the short exit called "TakeProfitExit" was filled at 1884.00, but no Ask bars on the chart ever dropped to 1884.00.

How do you know this?

I know this because the Green Dots in post #1's 2.jpg show 1 Tick Ask Bars and none of them are are at price 1884.00 (the lowest they go is post #1's 2.jpg is 1884.25)

The Last.jpg, Ask.jpg and Bid.jpg attached to this post (post #24) show the extact graphic settings I used to create the chart in post #1's 2.jpg.

Regarding this video

However, as you have requested, below is a video of me testing your script.

http://screencast.com/t/qG02H8tBdqX

I see in the video that the Replay Data you are testing covers the date of 4/21/16-4/22/16.
Please download and use the Replay Data from this link https://www.dropbox.com/sh/oo95l3dy4...T6hQDccja?dl=0 instead. This replay data covers the date 2/17/16 and is the exact replay data I used in post #1.

Also, when you retest using the new Replay Data, please only look at the short exit that happens on 2/17/16 at 12:00:34 AM EST, since this is the 1884.00 execution displayed in post #1's 2.jpg (and mentioned above) where the issue exists.
Thanks again for the help.

[NinjaTrader_ChelseaB]

Matheyas5,

I was hoping to make testing this easier to save time.., this is why i attempted to create a script that is simpler and more clearly is able to demonstrate the behavior without any special setup.

You have sent ES 03-16 data and you want me to play this back on April 21st starting at midnight?
(To confirm this data is for the 21st starting at midnight and not for the 22nd, is this correct?)

The ES 03-16 rolled over to the 06-16 on March 10th, 30 days before April 21st.
Are you certain this is the data you would like me to play back on April 21st?

Attached is a video of trying to test this data on April 21st and on April 22nd.

http://screencast.com/t/nczsaNo4

[Matheyas5]

You have sent ES 03-16 data and you want me to play this back on April 21st starting at midnight?
(To confirm this data is for the 21st starting at midnight and not for the 22nd, is this correct?)

No, this is not correct. The only date/time you need to Replay is 2/17/16 12:00:00AM EST to 2/17/16 12:01:00AM EST (which equals a total of 1 minute) and the only execution you need to look at is the short exit that happens on 2/17/16 at 12:00:34 AM EST.
I thought the quote below from post 24# made this clear, apologies if it wasn't clear. You'll also see a dropbox link in the quote below where you can download the exact 2/17/16 Replay data I used in post #1.

I see in the video that the Replay Data you are testing covers the date of 4/21/16-4/22/16.
Please download and use the Replay Data from this link https://www.dropbox.com/sh/oo95l3dy4...T6hQDccja?dl=0 instead. This replay data covers the date 2/17/16 and is the exact replay data I used in post #1.

Also, when you retest using the new Replay Data, please only look at the short exit that happens on 2/17/16 at 12:00:34 AM EST, since this is the 1884.00 execution displayed in post #1's 2.jpg (and mentioned above) where the issue exists.
Thanks again for the help.

[NinjaTrader_ChelseaB]

Hi Matheyas5,

Below is a link to a video of this test with the data on Feb 27th, 2016. My video is in mountain time so the time will be shown at 10:00 PM on Feb 26th instead of 12:00 AM on Feb 27th. The order in question is the same.

http://screencast.com/t/j3k6MsTRpLU

I am showing there were two ticks shown at the same time and the lower tick is the tick that filled the order.

[Matheyas5]

Hi Chelsea,

Did you use the Replay data I included in my dropbox link?

I ask because I'm wondering why you did not get an exit fill at 2/16/16 at 10:00:34 PM Mountain Time (AKA: 2/17/16 at 12:00:34 AM EST) like I did.

The above mentioned exit fill is the one that had the issue.

The other exit fills that I checked did not have the issue (however, I did not check all of them).

Thanks

[NinjaTrader_ChelseaB]

Hello Matheyas5,

I did use only the data that you provided.

The difference might be between our PC clocks. NT is sensitive to small time changes.

However, after the video I ran this through all of the data and examined every order. Either two ticks are received and typically the lower tick fills the order, or the order filled with the single tick available. I was not able to capture any fills that were not at a specific tick.

[Matheyas5]

Its not working that way on my PC. Can you please do a remote login on my PC to examine the issue?
Please private message me time your avail to do it if so. Thanks