AvgProfit == Average Net Profit?

[NinjaTrader_ChelseaB]

Hi Aventeren,

It is not possible to set the commission amounts using code.
However, I will submit a feature request in your behalf to our development department for this.

Thank you for your feedback and suggestions. We value your opinions and they will be considered in our feature discussion for a future NinjaTrader release.

I did want to mention that it would seem odd to me to apply slippage to a limit order. A limit order would not be affected by slippage. However, the idea sounds good.

[aventeren]

So I've been thinking about this commission and slippage issue, and another way to assume slippage for backtest purposes may be to just hardcode in the slippage and commission costs within the Tools > Options > Commissions > Futures - Simulator > Commission levels.

For instance, let's say that you use a $2.50/side commission, and that you have plugged that number into the Futures - Simulator Commissions value. But, you also want to assume that on average you are also going to incur 3 ticks of roundtrip slippage. Therefore, what you may consider doing would be to calculate the instrument specific cost of 1.5 ticks, and then just add this to the $2.50 Futures - Simulator Commissions value.

For example, let's say that you're backtesting the ES contract and that you are assuming $2.50/side in commission and 3 ticks roundtrip slippage. 3 ticks * $12.50/tick = $37.50 slippage. Therefore, you would divide $37.50 in half ($37.50 / 2 = $18.75) and add that number to $2.50 ($2.50 + $18.75 = $21.25) in the Commissions value, and you then have a net backtest result regardless of what order type you elect to use in your backtest (i.e., market or limit).

Based on the above, is it possible to define the Futures - Simulator Commission value within Initialize()? For instance, the above ES example would be $21.25, but a similar CL backtest would be $2.50 + (3 ticks * $10/tick)/2 = $17.50. Therefore, to run an ES backtest and then a CL backtest, one would need to remember to go back into the Futures - Simulator option and change the Commission value from $21.25 to $17.50. Instead, would it be possible to just have an if or switch statement that set the Commission value within the code so that you would not have to go to the extra step of clicking through screens to change the Commission value?

Thanks,

Aventeren

[aventeren]

Hi averteren,



You are likely referring to how the Default fill type requires that the price passes through a limit for it to fill.
Try using the Liberal fill type instead. This fills limits when they are touched and does not require a pass through. In a way that is more like the real market while the Default fill type is more like a worst case scenario.

Market / Stop Market orders will be filled at the open price of the next bar every time.

Yes, it does seem likely that a limit order entry and market order exit would result in 3 ticks of slippage. But I recommend that you test this and confirm.

Yes, I can confirm (through testing) that the market order will have slippage applied (within the price range of the bar) using the Default or Liberal fill types and that limit orders will need to be passed through when using the Default fill type.

Thanks, Chelsea.

I understand where you are coming from on the Liberal vs Default fill type (as it relates to limits being executed when touched or when penetrated, respectively), but I think I'm going to stick with the Default fill type so that this portion of my back test will have the potential to under-promise and over-deliver (i.e., guaranteed that the entry price would have been achieved) rather than over-promise and under-deliver (i.e., live miss an entry b/c price moved away).

I'll make sure to test the limit and stop = 3 ticks slippage. That might be a decent way to test to 3 ticks.

On the entries, your answer sheds light on why others use a secondary data series (i.e., 1 range) for entries and exits. Clearly using the secondary series for exits makes intuitive sense for back testing, but until you pointed out that a market order is not assumed to be executed until the open of the next bar, I did not understand why others used a secondary series for order entries. Now (I think) I do. For example, if you were back testing a daily bar and market entry order, then you would have to assume that if the OHLC met your entry condition that the assumed entry price would not be until the open of the next bar. But instead, if you had a piece of logic that first identified if the OHLC of the bar triggered the entry, which then dropped you into a secondary series to simulate the entry price, a better simulation would occur if you were planning on using COBC = false for your actual live testing. If you were using only going to be using COBC = true, then it likely wouldn't matter, as a live order would not be triggered until the open of the next bar anyway. I'll have to think about this...

Thanks for your help.

All best,

Aventeren

[NinjaTrader_ChelseaB]

Hi averteren,

From what I've read, I've been advised against using limit orders in backtesting because they generate unrealistic results. However, the following might produce reasonable back testing results:

1. Modify the strategy for back testing purposes to enter with limit orders (i.e., EnterLongLimit() and EnterShortLimit()) and exit with market orders (i.e., ExitLong() and ExitShort()).

2. Select "Default" in the "Fill type" parameter in the Strategy Analyzer dialog box, which amongst other things specifies that limit orders will only fill if penetrated. The documentation does not say anything about how the Default Historical Fill Processing in Operations > Strategy Analyzer > Understanding historical processing options > Historical Fill Processing handles market orders, but I'm assuming that market orders are considered executed if touched (can you please confirm this)?

3. Enter 3 for the Slippage value in the Strategy Analyzer dialog box, such that the back test engine will then add 0 ticks Slippage to the limit order and 3 ticks Slippage to the market (stop) order.

Does that seem likely to produce a back test with 3 ticks of roundtrip Slippage?

Also, can you please confirm with the Default Fill Type in the Historical Fill Processing dialog box that although limit orders will only be filled if penetrated, that market orders will still be filled with the specified Slippage when touched?

Thanks,

Aventeren

You are likely referring to how the Default fill type requires that the price passes through a limit for it to fill.
Try using the Liberal fill type instead. This fills limits when they are touched and does not require a pass through. In a way that is more like the real market while the Default fill type is more like a worst case scenario.

Market / Stop Market orders will be filled at the open price of the next bar every time.

Yes, it does seem likely that a limit order entry and market order exit would result in 3 ticks of slippage. But I recommend that you test this and confirm.

Yes, I can confirm (through testing) that the market order will have slippage applied (within the price range of the bar) using the Default or Liberal fill types and that limit orders will need to be passed through when using the Default fill type.

[aventeren]

Hi aventeren,

You could try using 1.5 but I'm not going to recommend it.

If you enter with a limit and exit with a market or stop market then only the market will have the slippage and you could use 3.

From what I've read, I've been advised against using limit orders in backtesting because they generate unrealistic results. However, the following might produce reasonable back testing results:

1. Modify the strategy for back testing purposes to enter with limit orders (i.e., EnterLongLimit() and EnterShortLimit()) and exit with market orders (i.e., ExitLong() and ExitShort()).

2. Select "Default" in the "Fill type" parameter in the Strategy Analyzer dialog box, which amongst other things specifies that limit orders will only fill if penetrated. The documentation does not say anything about how the Default Historical Fill Processing in Operations > Strategy Analyzer > Understanding historical processing options > Historical Fill Processing handles market orders, but I'm assuming that market orders are considered executed if touched (can you please confirm this)?

3. Enter 3 for the Slippage value in the Strategy Analyzer dialog box, such that the back test engine will then add 0 ticks Slippage to the limit order and 3 ticks Slippage to the market (stop) order.

Does that seem likely to produce a back test with 3 ticks of roundtrip Slippage?

Also, can you please confirm with the Default Fill Type in the Historical Fill Processing dialog box that although limit orders will only be filled if penetrated, that market orders will still be filled with the specified Slippage when touched?

Thanks,

Aventeren

[NinjaTrader_ChelseaB]

Hi aventeren,

You could try using 1.5 but I'm not going to recommend it.

If you enter with a limit and exit with a market or stop market then only the market will have the slippage and you could use 3.

[aventeren]

Excellent, this is proving to be super helpful.

So on Slippage, is there any way to generate a backtest for 3 ticks of roundtrip Slippage? If my understanding of your Slippage explanation (below) is correct, one could only have 0, 2, 4, 6, etc roundtrip Slippage ticks because the Slippage value in the Strategy Analyzer dialog box is per side and needs to be an integer (i.e., 0, 1, 2, 3, etc)?

Thanks, Aventeren

[NinjaTrader_ChelseaB]

Hi aventeren,

At an absolute minimum, I would STRONGLY suggest that you revise the .AvgProfit, .GrossProfit and .GrossLoss documentation sections to reflect the concepts in our discussion--at least an update for the NT8 documentation. In any event, that is what it is.

I'll send a message to our development, first letting them know of my test results, and to request to have the documentation edited to inform about the commissions.

a: Include a commission value defined under the Tools > Options > Commission > Futures - Simulator parameter,

b: Check the "Apply commission to PnL calculations"box in the Tools > Options > General

c: Set "Include Commissions" to true in the Strategy Analyzer dialog box, and

d: Enter a Slippage value in the "Historical Fill Processing" line in the Strategy Analyzer dialog box.

Yes, you do have the steps correct to include commission and slippage with backtest results.

I still don't have a firm handle on how NT deals with a non integer Slippage value in the Strategy Analyzer dialog box. The NT7 documentation page at Operations > Strategy Analyzer > Understanding backtest options > Slippage states that the value entered "Sets the slippage amount in ticks per execution". However, how does the back end code actually work? For instance, how would the NT Strategy Analyzer code deal with the following Slippage values:

1. Slippage = 1.01
2. Slippage = 1.49
3. Slippage = 1.50
4. Slippage = 1.51
5. Slippage = 1.99

How does the back test engine round Slippage to generate a result that has been rounded to TickSize? Is the engine simply rounding (i.e., 1.01 to 1.49 becomes 1, 1.51 to 1.99 becomes 2) or is the rounding always down (i.e., 1.01 to 1.99 becomes 1).

Also, how does the code process these double values? Is Slippage first applied to the entry and then the exit or just once at the exit by multiplying Slippage by 2?

Finally, if one was to set a Slippage value within the strategy's Initialize() method (as I showed before--let's assume a value of 2 here) but have a different value for Slippage in the Strategy Analyzer dialog box (let's assume a value of 1 here), which Slippage value is used by the engine code (i.e., 2 or 1).

The slippage (whole number in ticks) will be multiplied by the ticksize from the master instrument settings.

If the ticksize is .25 and slippage is 2, price slippage will be .50.
This needs to be a whole number as it is defining a number of ticks.

However, I am seeing that the number is not an integer, it is actually a double.
I checked without our development department about this and they said they would change the documentation but won't be changing the variable type in NinjaTrader.

I still highly recommend you keep this a whole number. I think you would get arbitrary values that do not relate to an actual tick value if you use a decimal.

Regarding setting the property in the code vs the strategy analyzer, the strategy analyzer will overwrite this. The property in the code will set the default that the strategy analyzer will be set to for this. So if you set it to 3 in the code it will be 3 by default when you run in the strategy analyzer.

[aventeren]

Hi aventeren,

To be honest, I wasn't sure about the commissions with gross profit / net profit and have been asking our development department what the expected behavior is and they have said to me that the gross profit / loss does not include commissions and this is what I have reported back to you.

After your further questions I decided to do a test to get definitive proof of the behavior and have found that this was incorrect.

Gross profit and loss does actually include commisssions in both the code and Strategy Analyzer results.

Attached are my test results and the strategy I used to test this. I tested this over ES 03-14 60 Minute from 2/18/2014 to 2/18/2014.

The results from output window with Include Comission set to True:
AvgProfit: 437.5
Gross Profit: 437
Gross Loss: 0
Total Commission: 0.5

Results with Include Commission set to False:
AvgProfit: 437.5
Gross Profit: 437.5
Gross Loss: 0
Total Commission: 0

So I was wrong about this. Commissions are included.

Chelsea--

I can't tell you how much I appreciate your follow through on my questions. I'm impressed. Thanks. No exclamation points. Just simply thank you.

I'm currently working to make sure that I clearly understand the back test engine before back testing, as if I don't understand the engine, then how can I possibly understand and have confidence in the results. So your answers to my questions are helping me to reach that level of confidence that I'm seeking.

At an absolute minimum, I would STRONGLY suggest that you revise the .AvgProfit, .GrossProfit and .GrossLoss documentation sections to reflect the concepts in our discussion--at least an update for the NT8 documentation. In any event, that is what it is.

So, can you please confirm the following:

1. For a Strategy Analyzer run to include commission and slippage within the resulting .AvgProfit, .GrossProfit and .GrossLoss values, the following must be done:

a: Include a commission value defined under the Tools > Options > Commission > Futures - Simulator parameter,

b: Check the "Apply commission to PnL calculations"box in the Tools > Options > General

c: Set "Include Commissions" to true in the Strategy Analyzer dialog box, and

d: Enter a Slippage value in the "Historical Fill Processing" line in the Strategy Analyzer dialog box.

I still don't have a firm handle on how NT deals with a non integer Slippage value in the Strategy Analyzer dialog box. The NT7 documentation page at Operations > Strategy Analyzer > Understanding backtest options > Slippage states that the value entered "Sets the slippage amount in ticks per execution". However, how does the back end code actually work? For instance, how would the NT Strategy Analyzer code deal with the following Slippage values:

1. Slippage = 1.01
2. Slippage = 1.49
3. Slippage = 1.50
4. Slippage = 1.51
5. Slippage = 1.99

How does the back test engine round Slippage to generate a result that has been rounded to TickSize? Is the engine simply rounding (i.e., 1.01 to 1.49 becomes 1, 1.51 to 1.99 becomes 2) or is the rounding always down (i.e., 1.01 to 1.99 becomes 1).

Also, how does the code process these double values? Is Slippage first applied to the entry and then the exit or just once at the exit by multiplying Slippage by 2?

Finally, if one was to set a Slippage value within the strategy's Initialize() method (as I showed before--let's assume a value of 2 here) but have a different value for Slippage in the Strategy Analyzer dialog box (let's assume a value of 1 here), which Slippage value is used by the engine code (i.e., 2 or 1).

My hope is that others will come across this post and come away with a better understanding of the NT7 Strategy Analyzer results.

Thanks for your help,

Aventeren

[NinjaTrader_ChelseaB]

Hi aventeren,

To be honest, I wasn't sure about the commissions with gross profit / net profit and have been asking our development department what the expected behavior is and they have said to me that the gross profit / loss does not include commissions and this is what I have reported back to you.

After your further questions I decided to do a test to get definitive proof of the behavior and have found that this was incorrect.

Gross profit and loss does actually include commisssions in both the code and Strategy Analyzer results.

Attached are my test results and the strategy I used to test this. I tested this over ES 03-14 60 Minute from 2/18/2014 to 2/18/2014.

The results from output window with Include Comission set to True:
AvgProfit: 437.5
Gross Profit: 437
Gross Loss: 0
Total Commission: 0.5

Results with Include Commission set to False:
AvgProfit: 437.5
Gross Profit: 437.5
Gross Loss: 0
Total Commission: 0

So I was wrong about this. Commissions are included.

[aventeren]

MaxNetProfit Optimize Type - No Commission?

Also, to confirm, the @MaxNetProfit.cs optimizer type (which can be selected in the "Optimize on..." line in the Strategy Analyzer dialog box) does not include commission?

Based on the code located in NinjaTrader 7 > bin > Custom > Type (below), it looks like slippage would be included (assuming that one entered a Slippage value and selected True for "Include Commission"), but that commission would not--in which case your definition of "net profit" is only net of slippage and not commission (which would probably be a more accurate representative of "net profit"). Correct?

// 
// Copyright (C) 2006, NinjaTrader LLC <www.ninjatrader.com>.
//
#region Using declarations
using System;
using System.ComponentModel;
using System.Drawing;
using NinjaTrader.Cbi;
using NinjaTrader.Data;
using NinjaTrader.Indicator;
using NinjaTrader.Strategy;
#endregion

// This namespace holds all strategies and is required. Do not change it.
namespace NinjaTrader.Strategy
{
	/// <summary>
	/// </summary>
	[Gui.Design.DisplayName("max. net profit")]
	public class MaxNetProfit : OptimizationType
	{
		/// <summary>
		/// Return the performance value of a backtesting result.
		/// </summary>
		/// <param name="systemPerformance"></param>
		/// <returns></returns>
		public override double GetPerformanceValue(SystemPerformance systemPerformance)
		{
			return systemPerformance.AllTrades.TradesPerformance.GrossProfit + systemPerformance.AllTrades.TradesPerformance.GrossLoss;
		}
	}
}

[aventeren]

Hi aventeren,

As long as Include commission is true in the backtest parameters, the slippage will be included with AvgPrice and GrossProfit.

The commission will not be included with the AvgPrice or GrossProfit. You'll need to add the commission with TradesPerformance.Commission.

Okay, great--I think we're getting closer.

Can you please confirm that my understanding is correct:

1. For a Strategy Analyzer run to include slippage within the resulting .GrossProfit and .GrossLoss values, one needs to enter a Slippage value in the "Historical Fill Processing" line and make sure that "Include Commissions" is set to true in the dialog box prior to running the backtest, optimization or walk forward optimization?

2. NT7 will not output a net profit figure, therefore you have to manually calculate net profit by .GrossProfit + .GrossLoss - .Commission?

3. From my understanding, entering a value of 2 for Slippage in the dialog box is a single side slippage value that reduces the .GrossProfit or .GrossLoss by 2 ticks/side * 2 sides = 4 ticks. However, how does NT7 handle this value of 2? For instance, if I wanted 3 round trip ticks to of assumed slippage, does NT7 wait to apply the slippage until the .GrossProfit and .GrossLoss values are calculated, or would it instead try and apply 1.5 ticks to each side, which is impossible due to rounding errors. How does NT7 handle fractional slippage values--and furthermore, how does one insure that an odd tick number of slippage is included?

Thanks,

Aventeren

[NinjaTrader_ChelseaB]

Hi aventeren,

As long as Include commission is true in the backtest parameters, the slippage will be included with AvgPrice and GrossProfit.

The commission will not be included with the AvgPrice or GrossProfit. You'll need to add the commission with TradesPerformance.Commission.

[aventeren]

Hi aventeren,

I understand now.

The slippage works with historical trades and does not work with real time trades.

This affects the fill type and where the trade exits. Basically it will add 2 ticks of loss to the exit price.

The default fill type code comments mention this will only affect market orders and stop market orders but will not affect limit orders.

Also, slippage can only be applied if the bar the order is filled on could have accommodated slippage.

Thanks, Chelsea.

As you might suspect, I'm asking these questions to understand how and where NT7 accounts for commission and slippage with the Strategy Analyzer's Optimization and Walk Forward Optimization.

Given the above, if I was to include $2.50 as the defined Commission under the Tools > Options > Commission > Futures - Simulator, check the "Apply commission to PnL calculations"box in the Tools > Options > General tab and then include 2 under the "Historical Fill Processing" within the Strategy Analyzer, will commission and slippage then be included in the backtested, optimized or walk forward optimized results?

Also, under the above assumption, would the .AvgProfit, .GrossProfit and .GrossLoss then include commission and slippage?

As you can tell, I am trying to get to a net profit number to back test on--rather than a gross profit number.

Thanks,

Aventeren

[NinjaTrader_ChelseaB]

Hi aventeren,

I understand now.

The slippage works with historical trades and does not work with real time trades.

This affects the fill type and where the trade exits. Basically it will add 2 ticks of loss to the exit price.

The default fill type code comments mention this will only affect market orders and stop market orders but will not affect limit orders.

Also, slippage can only be applied if the bar the order is filled on could have accommodated slippage.