Market replay price process understanding.

[Dzammer]

If we review your picture in your first post, we can see that 50.12 and 50.11 both happen at 8:39:50 .

I understand all of this stuffs, but problem is not on where execution is shown, but how NT7 terminal have fill my limit order by price 50.12, if market have no this price after 50.11 (where condition was true)?

[NinjaTrader_JessicaP]

Let's add two more pieces of information, so we can know if, and if so exactly when, the price reached 50.12 after 50.11. Please make the following changes to your code :

[FONT=Courier New]
        /// <summary>
        /// Called on each bar update event (incoming tick)
        /// </summary>
        protected override void OnBarUpdate()
        {[B]
            // OnMarketData will only work with live data
            // If you were backtesting to get this before,
            // please use the Market Replay connection instead
            if (Historical) return;[/B]

            // make sure we are using the primary series
            if (BarsInProgress != 0) return;


            // Condition set 1
            if (Close[0] <= MyInput0
                && !entered)
            {
                entered = true;
                Print("OnBarUpdate: Time = " + Times[1][0].ToString("dd MMM yyyy HH:mm:ss.ffff") + " / Close = " + Close[0][/FONT][FONT=Courier New][B][FONT=Courier New] + " / Current Bar " + CurrentBar[/FONT][/B]);
                SubmitOrder(0, OrderAction.Sell, OrderType.Limit, 1, 50.12, 0, null, "Entry");
            }
        }


        // This gets triggered once a trade is placed
        protected override void OnExecution(IExecution execution)
        {
                Print(String.Format("OnExecution: [B]Execution [/B]Time = {0:dd MMM yyyy HH:mm:ss.ffff} / [B]Execution Price [/B]= {1:C} [B]/ Close = {2:C} / CurrentBar = {3}[/B]",  execution.Time, execution.Pric[B]e, Close[0], CurrentBar[/B][/FONT][FONT=Courier New]));[B]
                entered = false;[/B]
        }[/FONT][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New]

[/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT]// http://ninjatrader.com/support/helpGuides/nt7/onmarketdata.htm[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT]protected override void OnMarketData(MarketDataEventArgs e)[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT]{[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New] [/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT] if (entered && e.MarketDataType == MarketDataType.Last)[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New] [/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT] {[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New]   [/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT]   Print("OnMarketData: Time = " + e.Time.ToString("dd MMM yyyy HH:mm:ss.ffff") + " / New price = " + e.Price + " / Current Bar " + CurrentBar);[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New] [/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT] }[/FONT][/B][B][FONT=Courier New]
[/FONT][/B][B][FONT=Courier New][FONT=Courier New]        [/FONT]}[/FONT][/B]

If we do this we will be able to see each tick as it comes in after your trade is placed, and we will also know which bar your trade is filled on.

[Dzammer]

Hello, Jessica.

I have attach result.
How we can see, after price 50.11, we have now price 50.12, but order have execute by this price...

OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471347
OnMarketData: Time = 04 апр 2017 07:39:50.0000 / New price = 50,11 / Current Bar 471347
OnExecution: Execution Time = 04 апр 2017 07:39:50.0000 / Execution Price = 50,12 ? / Close = 50,11 ? / CurrentBar = 471347
OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471348
OnMarketData: Time = 04 апр 2017 07:39:50.0000 / New price = 50,11 / Current Bar 471348
OnExecution: Execution Time = 04 апр 2017 07:39:50.0000 / Execution Price = 50,12 ? / Close = 50,11 ? / CurrentBar = 471348
OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471349
OnMarketData: Time = 04 апр 2017 07:39:50.0000 / New price = 50,11 / Current Bar 471349
OnExecution: Execution Time = 04 апр 2017 07:39:50.0000 / Execution Price = 50,12 ? / Close = 50,11 ? / CurrentBar = 471349
OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471350
OnMarketData: Time = 04 апр 2017 07:39:50.0000 / New price = 50,11 / Current Bar 471350
OnExecution: Execution Time = 04 апр 2017 07:39:50.0000 / Execution Price = 50,12 ? / Close = 50,11 ? / CurrentBar = 471350
OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471351
OnMarketData: Time = 04 апр 2017 07:39:50.0000 / New price = 50,11 / Current Bar 471351
OnExecution: Execution Time = 04 апр 2017 07:39:50.0000 / Execution Price = 50,12 ? / Close = 50,11 ? / CurrentBar = 471351
OnBarUpdate: Time = 04 апр 2017 07:39:50.0000 / Close = 50,11 / Current Bar 471352

[NinjaTrader_JessicaP]

Would it be possible to answer some questions about how we are backtesting?

• I noticed you have an execution every single bar. Can you let me know which instrument, time period you have data downloaded for, and number of bars back we are testing?
• I am assuming this is over 1 minute bars as in the original picture, on April 4th 2017?
• I noticed in your OP that you are using CL-## . NinjaTrader's market replay servers do not offer continuous contract data for this instrument. Can you let us know where you received this data, and where I can find a copy I can use to test on my end?
  • If not, can you repeat this test with CL 06-17 or CL 05-17 data?
  
  

The answer to these two questions will make it easier to determine exactly why and how this happened. I look forward to assisting further.

[NinjaTrader_JessicaP]

I realize your time is valuable. In order to repair any potential bug in NinjaTrader, we need to give the developers a clear case that they can see on their end. We need to be able to make the behavior you are seeing occur under controlled conditions to be able to learn what needs to be changed. If other customers observe the same behavior you are seeing I will report back here. Please let us know if there are any other ways we can help.

[Dzammer]

• I noticed you have an execution every single bar. Can you let me know which instrument, time period you have data downloaded for, and number of bars back we are testing?

All this params no matter. In really trade I use M60 bars, in the test, that I present for you it 1 tick bars. And we testing only those part where we have price down from 50.12 to 50.11 and less.

• I am assuming this is over 1 minute bars as in the original picture, on April 4th 2017?

No. It only for visualizing.

• I noticed in your OP that you are using CL-## . NinjaTrader's market replay servers do not offer continuous contract data for this instrument. Can you let us know where you received this data, and where I can find a copy I can use to test on my end?

This data I received this data from my broker AMP. I have attached file with data and also at this link (because it large).

[NinjaTrader_JessicaP]

Thank you Dzammer. Would it be possible to generate some real output using market conditions you can describe to me? This will make it easier for me to see the same thing on my end that you are seeing on yours.

[Dzammer]

Thank you Dzammer. Would it be possible to generate some real output using market conditions you can describe to me? This will make it easier for me to see the same thing on my end that you are seeing on yours.

I am not understaund what you mean. In real trading, for this day (04.04.17) I have no trades. And it right. But in the same data, on market replay I fave trade that I described. I send you data from my NT7 database. You have script code. You can do this on your computer and see. Write me if I can help you more.

[NinjaTrader_JessicaP]

I appreciate your patience. I mentioned in post 17,

[FONT=Courier New]
            // OnMarketData will only work with live data
            // If you were backtesting to get this before,
            // please use the Market Replay connection instead
            if (Historical) return;[/FONT]

The data you gave us is historical data, not market replay data. Historical data has the extension .ntd and the data you sent has the extension .nrd . When I test with ES 06-17 Market Replay data I can't see the same thing on my end that you were seeing on yours. Additionally, while I understand you were attempting to make things easier to see, it is impossible for me to find a way to get your strategy to place a trade every bar. This is what I mean by sending us real output; output that you have modified in any way in an attempt to make a description easier will not help us see the same thing on our end that you are seeing on yours.

The line of code from post 17

[FONT=Courier New]if (Historical) return;[/FONT]

Was designed specifically to prevent you using historical data to attempt to reproduce this.

When you try to act on the advice I have given in this thread, please

• Add that line of code to your strategy at the beginning of OnBarUpdate
• Use Market Replay data
• Use a chart for testing
• Send us your unmodified output. Please do not attempt to do anything for visualizing or for any similar reason

This said, I realize your time is valuable. If for any reason you can not set aside the time to follow the steps in this post, please do not attempt any more testing on your end, and please do not post any more results. Instead, we can reset your database to resolve your query. If this is the case, please

• Shut down NinjaTrader 8
• Delete the contents of your (My) Documents\NinjaTrader 8\db\cache folder
• Copy your (My) Documents\NinjaTrader 8\db\NinjaTrader.sdf file to another location as a back-up
• Restart NinjaTrader holding down the ctrl key on your keyboard
• In the Tools -> Database menu, in the Reset DB section, check both boxes and press Reset, Yes, and OK
• In the Tools -> Database menu, in the Update Instruments section, check all the boxes and press Update
• Restart NinjaTrader 8

Whichever path you choose, please don't hesitate to reach out if there is any other way we can help.

[Dzammer]

The data you gave us is historical data, not market replay data. Historical data has the extension .ntd and the data you sent has the extension .nrd .

I gave you all data, other part was at this link



Here link with all "db" folder and script. So you can reproduce problem on you PC.

Additionally, while I understand you were attempting to make things easier to see

Not only to see, but for exclude all other things that can influence to the result. And I have same error (now you have all data for reproduce it).

Add that line of code to your strategy at the beginning of OnBarUpdate

I done it many time ago.

Shut down NinjaTrader 8

I use NT7, that you can see in the screens that I posted.

[Dzammer]

The same error in other ticker GC 04-16.
Look in the screen. In the code we have:

if (Close[0] <= MyInput0
				&& !entered)
            {
				entered = true;
                Print("OnBarUpdate: Time = " + Times[1][0].ToString("dd MMM yyyy HH:mm:ss.ffff") + " / Close = " + Close[0] + " / Current Bar " + CurrentBar);
                SubmitOrder(0, OrderAction.Sell, OrderType.Limit, 1, 1225.6, 0, null, "Entry");
            }

Where MyInput0 = 1225.0;
So, when price <= 1225.0 we send sell limit by 1225.6. But how we can see on the screen, there is a no price 1225.6 after 1225.

How I wrote here (post #5) I think that problem in the time. All ticks have the same time. And NT interpretive all ticks how one bar, so in one bar we have both prices 1225 and 1225.6. And in this "bar" our condition become true and NT fill by price, that we have in this "bar".

[NinjaTrader_JessicaP]

Thank you for pointing out the data behind the link was different from the data you attached directly in post 21. I also understand that I overlooked this being an NT7 strategy. I was able to load this data in the playback connection in NT7.

I believe I will need just one more piece of information before I can see the same thing on my end. I found a run of bars that seemed to match the one in your picture, but this did not trigger the same trades the same way with the attached strategy. Could you let me know what time zone you are in? Could you also try the attached strategy in the Market Replay connection on the 4th of April and confirm that you get the same trades when using the Market Replay connection?

[NinjaTrader_JessicaP]

I apologize but I missed your most recent reply as it was not up when I began replying.

You are correct that the underlying cause of this behavior must be multiple ticks having the same timestamp. This situation is detailed here.

http://ninjatrader.com/support/helpGuides/nt7/how_trade_executions_are_plott.htm

NinjaTrader 8 will grant more granular features surrounding this situation, including using a custom bars series for fill resolution with no additional code, and filling when limit orders are touched.

[Dzammer]

Hello.

Could you let me know what time zone you are in?

UTC+1

Could you also try the attached strategy in the Market Replay connection on the 4th of April and confirm that you get the same trades when using the Market Replay connection?

Strategy that you attached not working:
1. Because

public class Example1687462 : Strategy
    {
        private bool entered = false;

[B]        // You not set Unmanaged = true and CalculateOnBarClose = false
        protected override void Initialize()
        {
            Unmanaged = true;
            CalculateOnBarClose = false;
        }[/B]

        protected override void OnBarUpdate()
        {
            /[B]/You use price 1225, but this is price for GC (Second example)if (Close[0] <= 1225.0 && !entered)
            if (Close[0] <= 50.11 && !entered)[/B]
            {
                entered = true;
                Print("Time = " + Time[0] + " / Close = " + Close[0]);
                SubmitOrder(0, OrderAction.Sell, OrderType.Limit, 1, 50.12, 0, null, "Entry");
            }
        }
        protected override void OnExecution(IExecution execution)
        {
            Print(String.Format("OnExecution: Execution Time = {0:dd MMM yyyy HH:mm:ss.ffff} / Execution Price = {1:C} / Close = {2:C} / CurrentBar = {3}",  execution.Time, execution.Price, Close[0], CurrentBar));
            entered = false;
        }
        protected override void OnMarketData(MarketDataEventArgs e)
        {
            if (entered && e.MarketDataType == MarketDataType.Last)
            {
                Print("OnMarketData: Time = " + e.Time.ToString("dd MMM yyyy HH:mm:ss.ffff") + " / New price = " + e.Price + " / Current Bar " + CurrentBar);
            }
        }
    }

This code also execute the same error. You can copy&past it.

You are correct that the underlying cause of this behavior must be multiple ticks having the same timestamp. This situation is detailed here.

Yes. This situation is the same... But. My question is not "Why NT7 draw entry in first tick", but "WHY NT7 FILLED ORDER IF WE HAVE NOT PRICE 50.12 AFTER 50.11 (where condition become true)???".

So, my question still opened.

[NinjaTrader_JessicaP]

Thank you again for bringing this behavior to our attention. As NinjaTrader 7 is no longer under active development, we will be using the information you have provided us to revise our documentation so that other users are aware of this corner case in backtesting. Thank you for using NinjaTrader 7.