Help backtesting strategy with intrabar granularity

[nrgtrader]

Having read http://ninjatrader.com/support/forum...ead.php?t=6652 on Backtesting NinjaScript Strategies with an intrabar granularity, I cannot seem to get the desired results...

I've added:

        protected override void Initialize()
        {
            Add(PeriodType.Tick, 1);     
            CalculateOnBarClose = true;            
        }

        protected override void OnBarUpdate()
        {            
            if (BarsInProgress == 0)
            {...        

            if (...                        
                    {
                    EnterLongLimit(1, true, 1, longEntryPrice, "Long");
                    }        
            else
            {
                return;
            }                    
        }

See the attached image to illustrate the problem... The order was entered on a limit, and exits for a profit on the same bar, even though price never achieved the needed levels when viewed on higher resolution.... the backtester is still referencing the primary data and not higher resolution data. Any help greatly appreciated!

[NinjaTrader_ChelseaB]

Hello nrgtrader,

If the script did not have intra-bar granularity, it would not be likely the entry and exit would be on the same bar.

Adding intra-bar granularity allows for actions to take place within a bar.

It appears that the limit is within the high and low of the bar and does have a valid price to fill with.

Do you have any prints or other information to show that the price did not reach the price of the limit?

What is the price of the limit?

What was the highest priced reached while that bar was open?

[nrgtrader]

ChelseaB thanks for following up. I double checked my code and re-ran the backtest... Same results. I also pulled up a 10 tick chart with the larger time-frame bar ends marked. Theres no way it could have been filled as indicated on the chart... Included is also the trace order log and order info.t appears to be referencing the correct series for the orders. Do you have any ideas?

CL 01-17 Buy 1 45.43 11/29/16 9:28 NT-00046 Entry 1L NT-00070 LongSignalEntry 0 1 Backtest
CL 01-17 Sell 1 45.58 11/29/16 9:28 NT-00047 Exit - NT-00072 Profit target 0 1 Backtest

11/29/2016 9:28:06 AM Entered internal PlaceOrder() method at 11/29/2016 9:28:06 AM: BarsInProgress=1 Action=Buy OrderType=Limit Quantity=1 LimitPrice=45.43 StopPrice=0 SignalName='LongSignalEntry' FromEntrySignal=''
11/29/2016 9:28:06 AM Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='LongSignalEntry' Mode=Price Value=45.31 Currency=0 Simulated=False
11/29/2016 9:28:40 AM Cancelled pending exit order, since associated position is closed: Order='NT-00071/Backtest' Name='Stop loss' State=Working Instrument='CL 01-17' Action=Sell Limit price=0 Stop price=45.31 Quantity=1 Strategy='TSARSTRAT' Type=Stop Tif=Day Oco='NT-00046-8480' Filled=0 Fill price=0 Token='f965b56a86294e1d9d8c4e90e940f73a' Gtd='12/1/2099 12:00:00 AM'
11/29/2016 9:28:22 AM Cancelled OCO paired order: BarsInProgress=0: Order='NT-00071/Backtest' Name='Stop loss' State=Cancelled Instrument='CL 01-17' Action=Sell Limit price=0 Stop price=45.31 Quantity=1 Strategy='TSARSTRAT' Type=Stop Tif=Day Oco='NT-00046-8480' Filled=0 Fill price=0 Token='f965b56a86294e1d9d8c4e90e940f73a' Gtd='12/1/2099 12:00:00 AM'
11/29/2016 9:28:40 AM Cancelled expired order: BarsInProgress=1: Order='NT-00071/Backtest' Name='Stop loss' State=Cancelled Instrument='CL 01-17' Action=Sell Limit price=0 Stop price=45.31 Quantity=1 Strategy='TSARSTRAT' Type=Stop Tif=Day Oco='NT-00046-8480' Filled=0 Fill price=0 Token='f965b56a86294e1d9d8c4e90e940f73a' Gtd='12/1/2099 12:00:00 AM'

[NinjaTrader_ChelseaB]

Hello nrgtrader,

The order in question is the profit target, is this correct?

I'm wanting to know the time the profit target was submitted (a print of the order object in OnOrderUpdate will show this), and the time and price of the execution (a print of the execution object in OnExecution will show this).

I'm not able to see when the profit target was submitted or filled in the output.
What are you seeing in this output that shows the order market Profit Target should not have filled on that bar?

(With the prints, we can see if the order is submitted while that bar is open before it closes, and see if the price was reached after the order was submitted but before the bar closes by printing the price of each tick until the close of the bar)

[nrgtrader]

ChelseaB Thanks for the suggestions. I have since reloaded my data and now cannot replicate the problem!

The fills looked 'funny' to me, and when I drilled down to the 10 tick chart (Red OHLC) it seemed to show the orders would have been impossible to execute, as the entry was at the low of the candlestick bar in question and price never recovered to the profit target area during that bar's time frame (Marked with black lines, ending at 9:28:41)

I have also added the calls to print execution and order data so I'll keep a look out and be back if the issue resurface.

Thanks.

[nrgtrader]

NinjaTrader_ChelseaB I noticed that SetProfitTarget() and SetStopLoss() orders attach to the primary dataset (BarsInProgress=0) -- or perhaps I'm misreading this print. If this is the case, could that be the cause of the potential impossible profit target I seemed to witness?

11/28/2016 10:02:58 AM Cancelled OCO paired order: BarsInProgress=0: Order='NT-00035/Backtest' Name='Profit target' State=Cancelled Instrument='CL 01-17' Action=BuyToCover Limit price=46.58 Stop price=0 Quantity=1 Strategy='TSARSTRATDEV' Type=Limit Tif=Day Oco='NT-00020-4518' Filled=0 Fill price=0 Token='37a3c0a7f6444667a2d2156f602ba6d5' Gtd='12/1/2099 12:00:00 AM'

[NinjaTrader_ChelseaB]

Hello nrgtrader,

This is correct. SetStopLoss(), SetTrailStop(), and SetProfitTarget() use the primary series data and cannot use a secondary series for fill pricing information. The Set methods only calculate historically on the primary bar series and there is no option to submit them to the secondary series or any other than the primary bar series.

To use intra-bar granularity for exits, you must use Exit orders such as ExitLongLimit/ExitLongStopLimit..

[nrgtrader]

NinjaTrader_ChelseaB I'm still having some difficulty after converting to managed trades (so I could track both entry and exit on the tick series bars in progress). See the attached screenshot and order/execution outputs. I cant see how the sequence could have executed as the output or screenshot indicates... Please let me know your thoughts.

2/3/2016 8:58:49 AM Order='NT-00023/Backtest' Name='Long' State=PendingSubmit Instrument='CLM' Action=Buy Limit price=30.59 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9009f4fca2404c43b2d4bee09cc5d1ba' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 8:58:49 AM Order='NT-00023/Backtest' Name='Long' State=Accepted Instrument='CLM' Action=Buy Limit price=30.59 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9009f4fca2404c43b2d4bee09cc5d1ba' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 8:58:49 AM Order='NT-00023/Backtest' Name='Long' State=Working Instrument='CLM' Action=Buy Limit price=30.59 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9009f4fca2404c43b2d4bee09cc5d1ba' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00023/Backtest' Name='Long' State=Filled Instrument='CLM' Action=Buy Limit price=30.59 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=1 Fill price=30.59 Token='9009f4fca2404c43b2d4bee09cc5d1ba' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Execution='NT-00014' Instrument='CLM' Account='Backtest' Name='Long' Exchange=Default Price=30.59 Quantity=1 Market position=Long Commission=0 Order='NT-00023' Time='2/3/2016 9:01:23 AM'
2/3/2016 9:00:18 AM Order='NT-00024/Backtest' Name='Stop' State=PendingSubmit Instrument='CLM' Action=Sell Limit price=0 Stop price=30.49 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='2c3eeb5f1dfb426f989296d43c7e6d62' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00024/Backtest' Name='Stop' State=Accepted Instrument='CLM' Action=Sell Limit price=0 Stop price=30.49 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='2c3eeb5f1dfb426f989296d43c7e6d62' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00024/Backtest' Name='Stop' State=Working Instrument='CLM' Action=Sell Limit price=0 Stop price=30.49 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='2c3eeb5f1dfb426f989296d43c7e6d62' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00025/Backtest' Name='Target' State=PendingSubmit Instrument='CLM' Action=Sell Limit price=30.71 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='70c17d6066b84595b2f44be9e553d29f' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00025/Backtest' Name='Target' State=Accepted Instrument='CLM' Action=Sell Limit price=30.71 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='70c17d6066b84595b2f44be9e553d29f' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00025/Backtest' Name='Target' State=Working Instrument='CLM' Action=Sell Limit price=30.71 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='70c17d6066b84595b2f44be9e553d29f' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00024/Backtest' Name='Stop' State=PendingCancel Instrument='CLM' Action=Sell Limit price=0 Stop price=30.49 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='2c3eeb5f1dfb426f989296d43c7e6d62' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00024/Backtest' Name='Stop' State=Cancelled Instrument='CLM' Action=Sell Limit price=0 Stop price=30.49 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='2c3eeb5f1dfb426f989296d43c7e6d62' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:00:18 AM Order='NT-00025/Backtest' Name='Target' State=Filled Instrument='CLM' Action=Sell Limit price=30.71 Stop price=0 Quantity=1 Strategy='MANAGEDSTRATEGY' Type=Limit Tif=Day Oco='' Filled=1 Fill price=30.71 Token='70c17d6066b84595b2f44be9e553d29f' Gtd='12/1/2099 12:00:00 AM'

[nrgtrader]

Please ignore last post! I was using the wrong Bars in Progress variable for trade management!

[nrgtrader]

NinjaTrader_ChelseaB I am still having problems after all. Attached is a screenshot of the trade, as well as the output. The signal fires at the bars in progress '0' close, I have CalculateOnBarClose = true yet the trade occurs within that bar's period.

I am using the managed approach, and all the bars in progress for order commands are using '1', for the 1-tick series.

Furthermore, the prints show the execution occurring at the same time at the close of the signal bar.

Am I missing anything?

2/3/2016 9:46:55 AM longSignal = True
2/3/2016 9:46:55 AM Order='NT-00019/Backtest' Name='Long' State=PendingSubmit Instrument='CLM 02-17' Action=Buy Limit price=30.83 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='52395b24ae4f4b36a8d1b77bbcba271d' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00019/Backtest' Name='Long' State=Accepted Instrument='CLM 02-17' Action=Buy Limit price=30.83 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='52395b24ae4f4b36a8d1b77bbcba271d' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00019/Backtest' Name='Long' State=Working Instrument='CLM 02-17' Action=Buy Limit price=30.83 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='52395b24ae4f4b36a8d1b77bbcba271d' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00019/Backtest' Name='Long' State=Filled Instrument='CLM 02-17' Action=Buy Limit price=30.83 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=1 Fill price=30.83 Token='52395b24ae4f4b36a8d1b77bbcba271d' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Execution='NT-00012' Instrument='CLM 02-17' Account='Backtest' Name='Long' Exchange=Default Price=30.83 Quantity=1 Market position=Long Commission=0 Order='NT-00019' Time='2/3/2016 9:46:55 AM'
2/3/2016 9:46:55 AM Order='NT-00020/Backtest' Name='Stop' State=PendingSubmit Instrument='CLM 02-17' Action=Sell Limit price=0 Stop price=30.73 Quantity=1 Strategy='STRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='d74f349fa8484cf7a4cb13507eef40fa' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00020/Backtest' Name='Stop' State=Accepted Instrument='CLM 02-17' Action=Sell Limit price=0 Stop price=30.73 Quantity=1 Strategy='STRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='d74f349fa8484cf7a4cb13507eef40fa' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00020/Backtest' Name='Stop' State=Working Instrument='CLM 02-17' Action=Sell Limit price=0 Stop price=30.73 Quantity=1 Strategy='STRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='d74f349fa8484cf7a4cb13507eef40fa' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00021/Backtest' Name='Target' State=PendingSubmit Instrument='CLM 02-17' Action=Sell Limit price=30.95 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9586058ba9c44abb8984a65b7e042cf4' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00021/Backtest' Name='Target' State=Accepted Instrument='CLM 02-17' Action=Sell Limit price=30.95 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9586058ba9c44abb8984a65b7e042cf4' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00021/Backtest' Name='Target' State=Working Instrument='CLM 02-17' Action=Sell Limit price=30.95 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=0 Fill price=0 Token='9586058ba9c44abb8984a65b7e042cf4' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00020/Backtest' Name='Stop' State=PendingCancel Instrument='CLM 02-17' Action=Sell Limit price=0 Stop price=30.73 Quantity=1 Strategy='STRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='d74f349fa8484cf7a4cb13507eef40fa' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00020/Backtest' Name='Stop' State=Cancelled Instrument='CLM 02-17' Action=Sell Limit price=0 Stop price=30.73 Quantity=1 Strategy='STRATEGY' Type=Stop Tif=Day Oco='' Filled=0 Fill price=0 Token='d74f349fa8484cf7a4cb13507eef40fa' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Order='NT-00021/Backtest' Name='Target' State=Filled Instrument='CLM 02-17' Action=Sell Limit price=30.95 Stop price=0 Quantity=1 Strategy='STRATEGY' Type=Limit Tif=Day Oco='' Filled=1 Fill price=30.95 Token='9586058ba9c44abb8984a65b7e042cf4' Gtd='12/1/2099 12:00:00 AM'
2/3/2016 9:46:55 AM Execution='NT-00013' Instrument='CLM 02-17' Account='Backtest' Name='Target' Exchange=Default Price=30.95 Quantity=1 Market position=Short Commission=0 Order='NT-00021' Time='2/3/2016 9:46:55 AM'

[NinjaTrader_ChelseaB]

Hello nrgtrader,

Is the entry order filled on the wrong bar?

Is the exit order filled on the wrong bar?

You've added a 1 tick series to the script, is this correct?
Are you waiting for the first tick of the new bar before submitting the exit?

(If you submit an order intra-bar that order can fill intra-bar. lf you don't want your exit until the next bar, wait until the next bar before placing the exit.)

[nrgtrader]

Hi NinjaTrader_ChelseaB. Yes, I've added the 1 tick series. I think your second question might indicate my problem. Do I need to have entry management in BarsInProgress == 1, even though that data series is indicated in the entry order itself?

The order entries are currently in (BarsInProgress == 0)...

Both entry and exit are wrong.

If I understand correctly, according to the example file Primary bars will always execute before the secondary bar series. So I'm wondering how its possible to have both orders occur at the same time as the signal which occurs on the primary times series...

[nrgtrader]

Entry conditions and orders are in (BarsInProgress == 0) for the example file...

[nrgtrader]

In case it wasn't clear, the signal occurred as it should OnBarUpdate for the primary (BarsInProgress == 0) data series. The problem is both the entry and exit orders also fired at that same time, which i don't understand (perhaps Ive coded it wrong).

The profit target and stop loss are triggered OnExecution of the entry (BarsInProgress == 1), which as I understand should have triggered the first tick after the close of (BarsInProgress == 0) series bar.

[NinjaTrader_ChelseaB]

Hello nrgtrader,

The order will be submitted when you want it to.

Both series will trigger OnBarUpdate.

If both series need to call at the same time, such as on the first tick of a new bar, the primary will likely trigger first and the tick series second.

When you mention the entry is incorrect, how is this incorrect?
What are you expecting?
Is the order placed after the primary bar closes and this is showing as filled on the trigger bar?
Is the order placed after the primary bar closes and this is showing on the next bar?
Is the order placed intra-bar and the order is showing as filled on the trigger bar?

Same for the exit. What is the behavior you are seeing, what is the behavior you are expecting?