Random results on every backtest

[corbeste]

Hi,

I have a very simple strategy to test that should enter/exit based on whether the current day is outside the previous days range - just getting used to the strategy analyser. However when I run it on the same data (tick data from 1st Jan - 18th Feb) the results are completely different each time. I notice that the MAE is huge (not respecting the stop loss), orders are cancelled randomly, and stop/profit targets are not honoured. Enter times are AFTER exit times on the Trades tab.

Is there some setting I am missing? If I close down and restart NT, it works fine for the first run, then the same random results come back. Is something cached?

[NinjaTrader_Bertrand]

Welcome to our forums corbeste - so you're only backtesting here or would an optimization via the Genetic Algorithm be included? This would not be an exhaustive search so not run through all paramters and thus could yield a different 'best' result when comparing multiple runs. Only the chart data would be cached by NT, not the actual backtest / SA run. For the ignored orders, please run with TraceOrders to have more order debug info available in the NT output window :

http://www.ninjatrader.com/support/forum/showthread.php?t=3627

[corbeste]

Hi,

I examined the output but it has not cleared anything up. I just want to understand how the attached image could occur - you can clearly see EntryTime is later than ExitTime. I am not clear if this is a coding error or my order management is completely messed up, but execution appears to be happening out of order.

Here is the relevant section from Output - notice how the date jumps from the 30th to the 14th Jan:

30/01/2013 08:45:00 Entered internal SetStopTarget() method: Type=Stop FromEntrySignal='target1' Mode=Price Value=141.75 Currency=0 Simulated=False
30/01/2013 08:51:00 Entered internal SetStopTarget() method: Type=Target FromEntrySignal='target1' Mode=Price Value=141.44 Currency=0 Simulated=False
30/01/2013 08:53:00 Entered internal PlaceOrder() method at 30/01/2013 08:53:00: BarsInProgress=0 Action=SellShort OrderType=Market Quantity=1 LimitPrice=0 StopPrice=0 SignalName='target1' FromEntrySignal=''
14/01/2013 15:08:00 Cancelled pending exit order, since associated position is closed: Order='NT-00051/Backtest' Name='Profit target' State=Working Instrument='FGBL ##-##' Action=BuyToCover Limit price=141.44 Stop price=0 Quantity=1 Strategy='GomProfileTSStratTest' Type=Limit Tif=Gtc Oco='NT-00032-386' Filled=0 Fill price=0 Token='753f2d4c59824737b0d1b56a53117720' Gtd='01/12/2099 00:00:00'
14/01/2013 15:08:00 Cancelled OCO paired order: BarsInProgress=0: Order='NT-00051/Backtest' Name='Profit target' State=Cancelled Instrument='FGBL ##-##' Action=BuyToCover Limit price=141.44 Stop price=0 Quantity=1 Strategy='GomProfileTSStratTest' Type=Limit Tif=Gtc Oco='NT-00032-386' Filled=0 Fill price=0 Token='753f2d4c59824737b0d1b56a53117720' Gtd='01/12/2099 00:00:00'

[NinjaTrader_Bertrand]

corbeste, is this a single or multi series script? Which exact NinjaScript methods are you using here?

[corbeste]

I assume its not a multi-series script because I dont know what that means I am just running a simple back-test on one instrument (Bunds).

The order entry is as follows:

SetStopLoss("target1",CalculationMode.Price,Close[0] - (stop*TickSize), false);
SetProfitTarget("target1",CalculationMode.Price,Cl ose[0] + (target1*TickSize));
EnterLong("target1");

(visa versa for shorts)

The init method:

protected override void Initialize()
{
CalculateOnBarClose = true;
EntriesPerDirection = 1;
EntryHandling = EntryHandling.UniqueEntries;
TraceOrders = true;
}

The OnBar Update method:

protected override void OnBarUpdate()
{
EntryHandling = EntryHandling.UniqueEntries;

if (Position.MarketPosition != MarketPosition.Flat) return;

if (CurrentBar > 1)
_gpb.Update(); (this calculates VWAP and POC via a indicator)

if ((Bars.GetDayBar(1) != null .....
<strat management, just a couple of simple logic statements to determine if I go long or short>
.

[NinjaTrader_Bertrand]

Thanks.

MultiSeries would mean you're adding another timeframe or instrument into the strategy, thus working the calculations / fills off multiple bar objects.

Do you reset the stops and target in OnBarUpdate() when in flat state?

http://www.ninjatrader.com/support/forum/showthread.php?t=3222

[corbeste]

The stoploss/targets are set using CalculationMode.Price (based on some tutorial I found on the net), so they are set each time a position is entered. I dont clear them in any other way, as I assumed they were OCO orders so if one was triggered, the other would cancel.

In my GoLong function I call the following each time to entry criteria is met AND the position is flat:

SetStopLoss("target1",CalculationMode.Price,Close[0] - (stop*TickSize), false); SetProfitTarget("target1",CalculationMode.Price,Cl ose[0] + (target1*TickSize));
EnterLong("target1");

This is all the order management I have coded, apart from having ExitOnClose = true in the backtest properties.

I still do not understand how this could lead to entries after exits.. it just doesn't make any sense and implies that Ninja has lost the ability to track the orders in some way or form during the backtest.

[NinjaTrader_Matthew]

The Orders themselves will cancel, but the actual price they're being submitted to would need to be reset.


Please see the following reference sample which demonstrates how to modify the price of a stop loss. You will find one of the first lines of OnBarUpdate() resets your stop loss value so it will be correctly calculated from the new entry point for each new position:

http://www.ninjatrader.com/support/forum/showthread.php?t=3222

[corbeste]

Thanks, this solved the problem.