set stop loss orders not working consistently.

[rtwave]

people with nt,





for a long time i have been trying to incorporate onorderupdate and onexecutionupdate to some strategies of mine to be able to have two separate entries and superior risk management. i have put a lot of effort into these capabilities but have never been able to make exitstopmarket orders work as intended. i have reported the problems i have had extensively:

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/1146777-strategies-with-one-market-and-one-stop-market-entry-orders

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/1166008-interested-in-collaboration-with-other-users-to-refine-some-code

in view of this setback, i decided to stop trying to develop strategies with two entries and to concentrate on strategies with one single entry and setstoploss orders. i had never had significant issues using these setstoploss orders but now i have also been getting endless inconsistencies on my backtests - optimizations.


















the stop loss order is set at $900 usd (plus slippage), so every trade that losses significantly more than this amount is a malfunction as these strategies do not hold positions when markets are closed.


i have no idea what could be causing these problems. it could be the platform, the data, i just don't know. i had ran identical optimizations a very large number of times in the past and results were good, usually there were 3 trades at most that lost more than what the stop loss order was set to and the bigger losses were still fairly close to the stop loss amount. i will run a backtest and upload the output logs as soon as possible, but other than that, ¿what other data - information could nt need to be able to solve these issues definitively? my strategies generate ordinary entries and exits on 5 minute bars and i also have a secondary 1 minute series to monitor trading hours, this should not be a problem but it seems like it could be.



very well, regards.

[NinjaTrader_Jim]

Hello rtwave,

Thanks for your post.

Order fill simulations with backtesting are based on the Open High, Low and Close values of the bar. I would normally suggest trying again with Order Fill Resolution set to High and use a single tick data series as the fill series, but this would not be possible as you have added a single tick data series to the script.

Some solutions to take:

1. Add a single tick data series to your script and use Exit methods so you can submit the orders to the single tick data series so you can achieve intrabar order fills.
2. Try removing the minute data series and modify the strategy so it is a Single Time Frame strategy, so you can use High Order Fill Resolution
3. Use Market Replay data with the Playback Connection for accurate testing, and accept that the Strategy Analyzer/historical processing will not be able to create accurate fills in the current setup.

Understanding Historical Fill Processing - https://ninjatrader.com/support/help...ical_fill_.htm

Backtesting with Intrabar granularity (This would be referenced if you want to use Exit methods submitting to a single tick data series) - https://ninjatrader.com/support/help...ipt_strate.htm

Playback - https://ninjatrader.com/support/help...l?playback.htm

Let us know if you haver any additional questions.

[rtwave]

people with nt,




i have now ran a backtest and saved all nt output to a file and have attached it to this post.


the entry times for the two worst trades are:


09/08/2018 18:02:00 10/08/2018 16:59:58
16/06/2021 18:02:00 17/06/2021 01:35:23


i ran this backtest on tick data with a secondary 1 minute series and there are more than 50 trades that lose more than what the stop loss order is set to (screengrab #4 in the previous post corresponds to this backtest).


i had not had this kind of malfunctions on strategies with one single entry and set orders. in fact, when working on strategies with two entries i created an experimental version with one entry and set orders and it worked perfectly:

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/1166008-interested-in-collaboration-with-other-users-to-refine-some-code?p=1166478#post1166478

other than this output, ¿what else would nt need to be able to solve this problem?



very well, regards.

[NinjaTrader_Jim]

Hello rtwave,

Could you create an example script that just contains enough code to demonstrate your issue? We would ask that example script does not have any input parameters, and the entry conditions are simplified (I.E. just check if the position is flat before entering.) Please then provide a screenshot showing the data series settings you used to test.

With an example like that we could better understand what exactly you are doing and can better explain the behavior.

https://ninjatrader.com/support/helpGuides/nt8/index.html?export.htm#ExportingninjascriptAsSourceFiles

[rtwave]

NinjaTrader_Jim,



thanks.




i have been able to quickly create a sample strategy and it fortunately (or unfortunately) does replicate the same malfunctions i have been running into.


i have uploaded it as an attachment.



i ran one optimization process and there were lots of trades which lost more than what the stop loss should have held those to.


all the settings to optimize or backtest this strategy and replicate these issues can be found on these following screengrabs.










possibly my installation of the nt platform has become corrupted, or maybe some database, i had not had this scale of malfunctions before, even when working with almost identical strategies.



and i have great interest in fixing this because if i am ever going to run strategies automated unattended overnight, i first have to make sure that risk and position management are flawless. setting hard stops in my strategies and then having lots of trades which run over those stops would mean that i have put a lot of effort into coding in nt and all for naught.



very well, regards.

[NinjaTrader_Jim]

Hello rtwave,

I was looking for a reduced example that just focuses on the stop loss behavior, and has the input parameters removed.

However, I do have enough information to help you further by testing just the parameters given in your screenshot.

If you right click on each trade that is greater than $900 in loss (180 ticks * 0.25 ticksize * 20 point value) and select Chart, you can see these trades take place where there are market gaps. Since the OHLC values of the bar are used to simulate order fills, we can expect to see fill price deviation your designated price levels for the stop loss.

[rtwave]

people with nt, NinjaTrader_Jim,



i have taken a look at the 3 largest losing trades. there are no gaps in the nq when those trades were calculated. furthermore the strategy can only open and hold positions between 18:00 and 16:00 hours, so a clean gap of tens of points in the nq during cme market hours is not something to be expected.




Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
842 NQ 12-21 Backtest tesampletrientscstlo Short 1 7802.75 8068.25 09/03/2020 18:02:00 10/03/2020 02:00:00 shretien01 Stop loss -$5,310.00 $19,030.00 $0.00 $6,600.00 $1,130.00 $6,440.00 2



Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
826 NQ 12-21 Backtest tesampletrientscstlo Short 1 8236.25 8463.25 01/03/2020 18:02:00 02/03/2020 02:00:00 shretien01 Stop loss -$4,540.00 $21,365.00 $0.00 $5,050.00 $1,600.00 $6,140.00 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
1200 NQ 12-21 Backtest tesampletrientscstlo Short 1 11228.00 11454.50 03/11/2020 18:02:00 04/11/2020 02:00:00 shretien01 Stop loss -$4,530.00 $71,570.00 $0.00 $4,530.00 $560.00 $5,090.00 2



the stop orders should be set 180 ticks or 45 points away from the entry price. i see no reason why the nt platform should not be able to place such an order and then act on it if triggered on any minute bars series.


something i have just noticed is that at least the 5 largest losing trades all have an entry time of 18:02, which is when this strategy should restore positions after the market reopens. and all have an exit time of 02:00, which must be some kind of glitch - bug.



Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
853 NQ 12-21 Backtest tesampletrientscstlo Short 1 6996.50 7218.75 16/03/2020 18:02:00 17/03/2020 02:00:00 shretien01 Stop loss -$4,445.00 $42,035.00 $0.00 $4,445.00 $1,820.00 $6,265.00 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
869 NQ 12-21 Backtest tesampletrientscstlo Short 1 7012.25 7183.50 23/03/2020 18:02:00 24/03/2020 02:00:00 shretien01 Stop loss -$3,425.00 $41,705.00 $0.00 $3,425.00 $825.00 $4,250.00 2



as i first stated, this is something that i had not had problems with before. in the case of strategies with two entries, i was mainly working with the rty and m2k contracts and even when the sizes of the stop loss orders were very different, i also had a lot of problems in those cases.



¿can anyone else replicate these abnormal results? ¿what possible causes could explain these malfunctions?

[rtwave]

people with nt,




i took a look at a different strategy, on a different instrument.



this is another single entry strategy which also uses setstoploss orders and the largest losing trades have an identical structure, all are opened at 18:02 and closed at 20:00. liquidated much lower than what the stop loss was set to.



Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
250 M2K 12-21 Backtest Long 1 1145.6 1118.2 15/03/2020 18:02:00 15/03/2020 20:00:00 retiloen01 Stop loss -$137.00 -$264.50 $0.00 $137.00 $105.50 $242.50 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
112 M2K 12-21 Backtest Long 1 1470.0 1451.0 09/10/2019 18:02:00 09/10/2019 20:00:00 retiloen01 orexretiloen01 -$95.00 $287.00 $0.00 $105.00 $0.50 $95.50 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
256 M2K 12-21 Backtest Long 1 1006.9 989.3 19/03/2020 18:02:00 19/03/2020 20:00:00 retiloen01 Stop loss -$88.00 $148.50 $0.00 $89.00 $15.50 $103.50 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
555 M2K 12-21 Backtest Long 1 1975.4 1963.5 20/12/2020 18:02:00 20/12/2020 20:00:00 retiloen01 orexretiloen01 -$59.50 $2,390.50 $0.00 $104.50 $3.00 $62.50 2


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
254 M2K 12-21 Backtest Long 1 1055.1 1043.3 17/03/2020 18:02:00 17/03/2020 20:00:00 retiloen01 Stop loss -$59.00 $51.50 $0.00 $95.00 $13.00 $72.00 2




and i decided to also take a look at the two entry strategies i have been trying to refine. this strategy is similar to the pullback sample strategy i have attached to the other threads i referenced in this post. this strategy uses exitstopmarket orders:


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
638 M2K 12-21 Backtest Long 1 1982.3 1930.5 04/01/2021 03:02:00 04/01/2021 12:24:00 lopuen01 loorexpuen01 -$777.00 $7,780.50 $0.00 $919.50 $111.00 $888.00 9


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
367 M2K 12-21 Backtest Long 1 1435.9 1391.2 27/05/2020 21:55:00 28/05/2020 16:44:00 lopuen01 locltiexpuen01 -$670.50 $5,545.00 $0.00 $862.50 $25.50 $696.00 17


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
242 M2K 12-21 Backtest Long 1 1507.0 1462.4 03/03/2020 03:38:00 03/03/2020 16:45:00 lopuen01 locltiexpuen01 -$669.00 $6.00 $0.00 $819.00 $376.50 $1,045.50 12


Trade number Instrument Account Strategy Market pos. Qty Entry price Exit price Entry time Exit time Entry name Exit name Profit Cum. net profit Commission MAE MFE ETD Bars
930 M2K 12-21 Backtest Long 1 2249.7 2208.0 14/09/2021 08:34:00 14/09/2021 15:51:00 lopuen01 loorexpuen01 -$625.50 $13,955.50 $0.00 $685.50 $52.50 $678.00 7




in this case, the situation is different, these are all pullback entries and the entry and exit times are all different. however, even when the malfunction is different, the end result is the same: losing trades multiple times larger what the stop loss should be (~$48 usd).

[NinjaTrader_Jim]

Hello rtwave,

I will be replying back later next week to go more into depth regarding the gaps seen. I return to the office on Tuesday.

As for the second matter, please attached a reduced test script and instruction to reproduce on our end. The test script should have user inputs removed so the script is more easily testable (and should in turn by shown with a regular backtest on a small set of data)

[rtwave]

i have done more testing and these are the results:


my sample strategy, optimized on 20210101 to date data, and there are multiple losing trades that are out of order.






i then created another strategy, i just took the entry - exit logic from the old, tired nt sample ma crossover strategy, placed it inside my strategy keeping the time filters in place and the results are even worse:






and finally, i evaluated this last strategy on just 4 months of data and there are still trades that lose far more than what the stop loss order should keep the maximum loss to:






not only are there multiple erroneous trades with the 18:02 to 02:00 hours sequence i have reported previously, there are also a large number of trades which the platform reports as having triggered the stop loss order even when the entry and exit times are the exact same. 45 points in the nq in a minute is not credible, but 45 points in less than a second is obviously wrong.



¿are there any modifications i can make to the stop orders so that they are observed on the secondary 1 minute series in my strategy?

[NinjaTrader_ChelseaB]

Hello rtwave,

The trades are currently sorted by the Profit column and are not sorted by the Trade number column. Is this causing confusion about the order of the trades in the window?

Focusing on the first trade, what is the entry price of the trade? On the Orders tab what is the Stop price the stop loss? (This is to see where the order was submitted vs where the order filled)
Take the entry minus the stop price price and divide by the TickSize. Was this more than the number of ticks supplied to SetStopLoss()?

Currently it appears this script is being back-tested without intra-bar granularity which can affect the order fill price accuracy.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live?p=773377#post773377

Please provide an export of the simplified test script that is able to reproduce.
To export a NinjaTrader 8 NinjaScript so this can be shared and imported by the recipient do the following:

1. Click Tools -> Export -> NinjaScript...
2. Click the 'add' link -> check the box(es) for the script(s) and reference(s) you want to include
3. Click the 'Export' button
4. Enter a unique name for the file in the value for 'File name:'
5. Choose a save location -> click Save
6. Click OK to clear the export location message

By default your exported file will be in the following location:

• (My) Documents/NinjaTrader 8/bin/Custom/ExportNinjaScript/<export_file_name.zip>

Below is a link to the help guide on Exporting NinjaScripts.
http://ninjatrader.com/support/helpG...-us/export.htm

Once exported, please attach the file as an attachment to your reply.

[rtwave]

people with nt,



i have already shared a strategy that will exhibit these malfunctions which consist of having most losing trades kept below a stop loss order but also several trades which lose far more than what the stop loss order should keep them to. the cs files can be found in this same thread in the post linked below and i have now uploaded a zip file again.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/1173922-set-stop-loss-orders-not-working-consistently?p=1174074#post1174074

i have become convinced that this code below is the problem.


i use larger bars to identify downtrends and uptrends, and a 1 minute data series to enforce time limits to trade inside of and also to liquidate and restore positions at the close and open. i would also like to have this 1 minute series monitor the stop loss orders and secondary entries, but it seems that nt cannot accomplish this.

else if(BarsInProgress == 1)
{

if ((Times[1][0].TimeOfDay >= Tiexnotr.TimeOfDay)
&& (Times[1][0].TimeOfDay <= Tisttrre.TimeOfDay))
{
notrco = true;
}
else
{
notrco = false;
}

if ((Times[1][0].TimeOfDay >= Tiexnotr.TimeOfDay)
&& (Times[1][0].TimeOfDay < Tiexnotr.AddMinutes(2).TimeOfDay)
&& (Position.MarketPosition == MarketPosition.Short))
{
ExitShort(Convert.ToInt32(Posi), @"shcltiexoren01", @"shoren01");
ExitShort(Convert.ToInt32(Posi), @"shcltiexretien01", @"shretien01");
}

if ((Times[1][0].TimeOfDay >= Tiexnotr.TimeOfDay)
&& (Times[1][0].TimeOfDay < Tiexnotr.AddMinutes(2).TimeOfDay)
&& (Position.MarketPosition == MarketPosition.Long))
{
ExitLong(Convert.ToInt32(Posi), @"locltiexoren01", @"looren01");
ExitLong(Convert.ToInt32(Posi), @"locltiexretien01", @"loretien01");
}

if ( (Times[1][0].TimeOfDay >= Tisttrre.TimeOfDay)
&& (Times[1][0].TimeOfDay < Tisttrre.AddMinutes(3).TimeOfDay)
&& ( dtc == true )
&& (Position.MarketPosition != MarketPosition.Short) )
{
EnterShort(Convert.ToInt32(Posi), @"shretien01");
}

if ( (Times[1][0].TimeOfDay >= Tisttrre.TimeOfDay)
&& (Times[1][0].TimeOfDay < Tisttrre.AddMinutes(3).TimeOfDay)
&& ( utc == true )
&& (Position.MarketPosition != MarketPosition.Long) )
{
EnterLong(Convert.ToInt32(Posi), @"loretien01");
}


}

i also had this code in the strategies with two entries that i haven't been able to make work, i'm glad that i haven't paid any programmer for their services as i'm now convinced that my code is not the problem.


i will not try to use a 1 tick data series instead of the 1 minute series. that has nothing to do with the issue at hand. intra bar granularity is not the problem either, i can use a 1 minute series for secondary entries without problem. and most importantly, in all the tests i have reported - most - of the losing trades are kept to a maximum amount by the stop loss orders, it is only some trades where the stop loss order fails to do its job as i have proved with all the screengrabs and code i have shared. if intra bar granularity was the issue, - none - of the trades in my tests would be liquidated at the desired amount and that is evidently not the case. if stop loss orders work well 70 to 90% percent of the time, but there are 10 to 30% of losing trades that can be left unchecked and go on to lose very large amounts i call that a very evident platform malfunction.







this above is the same sample strategy i have uploaded as an attachment, i just removed the secondary data series and the time limits and then - all - stop loss orders work to perfection. even when i am evaluating over data going as far back as 2019 to date.



time limits and liquidating - restoring positions is crucially important because most brokers have very sensitive margin requirements around the time when futures markets are closed. i will try to run some of my strategies in real time in simulator, if i conclude that nt cannot reliably have both time limits, secondary data series and stop loss orders in a strategy then i would never consider using this platform to trade automated with real funds. i have provided all the necessary evidence as well as also sample strategies to replicate these issues, it is up to the people at nt to acknowledge these malfunctions for what they are and to fix them.

[rtwave]

i have been able to develop strategies which look like this, even when some stop loss orders do not always work.


it would be awful if nt cannot be corrected to where it can backtest - optimize strategies which include secondary data series, secondary entries, time limits and stop loss orders, and also trade them automated without errors. but well, this is the situation, as things stand i cannot consider using nt to trade automated overnight unless all stop loss orders can be guaranteed to work as they are intended to.

[NinjaTrader_Jim]

Hello rtwave,

The test information given is not clear to demonstrate to us what a specific problem is. When demonstrating an issue, we ask you provide:

1. A small test script with user inputs removed and code simplified to demonstrate an issue with a specific trade
2. Steps/Instruction to reproduce with a small set of data, to look at a specific trade.

In this case, the screenshot you have given is of a modified strategy, and it is not a reduced strategy, or a specific test to demonstrate a specific issue.

We ask for this information as it demonstrates what specifically you expect to see and what looks incorrect, with specific, testable context.

All we have been shown so far is that it is not clear how the orders are filling with historical processing. This is the underlying issue, which we can explain.

Historical (backtest) orders fill based on the data series they are submitting to, and the bars of that data series. Looking into how the order fills the way it did involves understanding the bars used to fill that order.

When you have a multi time frame strategy, orders are filled based on the data series that the order was submitted to. This is either the data series in which the order submission method is called, or to the data series the order is submitted to when using a Multi Time Frame order method overload.

Example:

EnterLong(int barsInProgressIndex, int quantity, string signalName)
EnterLong(1, 1, "My Entry");

The above will have an order submitted to the first added data series, or in your case the 1 minute data series.

In your strategy, you are mixing entries and exits in different BarsInProgress indexes, so the data series used to fill these orders will be mixed.

https://ninjatrader.com/support/helpGuides/nt8/index.html?multi-time_frame__instruments.htm#EnteringExitingAndRetrievingPositionInformation

The solution I would recommend to your situation is using intrabar granularity. This does not just involve adding an additional data series. It involves using the order method overloads that allow specifying the BarsInProgress index for the data series you want to fill the order.

Attached is an example strategy, which is a modification of SampleOnOrderUpdate that implements intrabar granularity for backtesting. The main take away is that Enter/Exit methods are used, and overloads that let us specify a specific BarsInProgress are used. In the example, all orders are submitted to a 1 tick data series, which would be the most granular we can get for backtesting. You are welcome to modify that data series as a 1 minute data series, and this will show greater variance, but would be less resource intensive, especially for long backtests.

If you have questions on there being a problem with NinjaTrader or a question on this approach, please do so referencing the example strategy I have attached which follows practices to have consistently filled orders.

[rtwave]

NinjaTrader_Jim, people with nt,



i have been working with the intrabar granularity sample and little by little i have finally been able to make it perform as intended.



i have some questions and comments:


- i first tried to reference set stop orders to a secondary data series but it is impossible. it would be appropriate if it was clearly stated in the user guides that these orders will not work on multiple data series strategies as that is not mentioned anywhere.


- i imagine that trailing stop orders will not work on multiple data series strategies either, ¿is this correct?


- and the bars in progress terminology is not intuitive at all. i understood it as if bip == 0 was a binary variable where 0 meant the exact close of the larger bar (or index 0), thus not in progress, and == 1 meant that indeed the bar was in the progress of being drawn. if that particular terminology was changed to something like - data series index - that would be a very helpful.



now i will try to make some incremental improvements incorporating break even stop orders and if i can secondary entries. it is not easy but i will see whether i can make some further modifications.



very well, thanks, regards.