Some Limit Orders Not Placed in Backtest

[JagelBagel]

Hello friendly ninja forum folks,

I am having a very strange issue with some of my limit orders in my backtests of this strategy. The general logic is as follows:
1. Search for levels and add new ones to list
2. Place orders for levels that currently do not have an order attached
3. Profit

This is working great for most levels, but there are some (can't find a distinction to separate working from not) that simply don't get placed. Here is what I have tried and a bit of the code:

1. Enabled order traces.
An order that is not working:
Entered internal SubmitOrderManaged() method at 2024-09-26 7:51:08 AM: BarsInProgress=0 Action=SellShort OrderType=Limit Quantity=1 LimitPrice=20338.50 StopPrice=0 SignalName='ShortSignal20339.25' FromEntrySignal=''
An order that IS working:
Entered internal SubmitOrderManaged() method at 2024-09-26 7:51:08 AM: BarsInProgress=0 Action=Buy OrderType=Limit Quantity=1 LimitPrice=20209.00 StopPrice=0 SignalName='LongSignal20208.25' FromEntrySignal=''

2. Order state logging.
An order that is not working prints nothing.
An order that IS working prints:
Order submitted: LongSignal20208.25
Order accepted: LongSignal20208.25
Order working: LongSignal20208.25

3. Other logging.
Here is the relevant order management code (same for buys):

foreach (int levelIndex in masterSellEvaluated[1])
{
    try
    {
        PrintDebug("This is the master list: " + masterSellLevels);
        PrintDebug("LEVEL IN MASTER LIST: " + masterSellLevels[levelIndex]);
        masterSellLevels[levelIndex].order = EnterShortLimit(0, true, Quantity, masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize), masterSellLevels[levelIndex].signalName);
        if (masterSellLevels[levelIndex].order == null)
        {
            PrintError("abc123 FAILED AT: " + (masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize)));
        }
        else
        {
            PrintInfo("ENTERED SHORT LIMIT AT: " + (masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize)));
            PrintDebug("The level in the master list: name - " + masterSellLevels[levelIndex].signalName + " | order details - " + masterSellLevels[levelIndex].order + " , " + masterSellLevels[levelIndex].order.LimitPrice + " , " + masterSellLevels[levelIndex].order.OrderAction);
        }
    }
    catch (Exception e)
    {
        if(LogErrors)
            Print("Error submitting order: " + e.Message);
    }
}​

I'm not setting any exit orders or conditions other than EOD for testing. masterSellEvaluated[1] is a list of indexes for levels in the masterSellLevels list that need submitting.
Removal of expired levels comes after the submitting, so indexes are not broken. It is the correct level being modified.
The following is printed after the order trace message...
An order that is not working prints:
abc123 FAILED AT: 20338.5
An order that IS working prints:
ENTERED LONG LIMIT AT: 20209
The level in the master list: name - LongSignal20208.25 | order details - orderId='NT-00046-1028' account='Backtest' name='LongSignal20208.25' orderState=Working instrument='NQ 12-24' orderAction=Buy orderType='Limit' limitPrice=20209 stopPrice=0 quantity=1 tif=Gtc oco='' filled=0 averageFillPrice=0 onBehalfOf='' id=-1 time='2024-09-25 13:02:38' gtd='2099-12-01' statementDate='2024-10-08' , 20209 , Buy

I simply don't understand how I can explicitly set the order of a level and then in the next line of code it is null? Makes no sense when the order trace says that it got placed.
And no it does not have to do with the current price being unfit for limit orders. Close at placement of the working order was Close: 20224.5 and at non working order was Close: 20296.25
It is also not only sells that have the issue, both long and short have working and non working.

If anyone has any idea, please let me know. I am completely stumped and discouraged. Also, thanks for taking the time to read through and help me out <3

[NinjaTrader_Jesse]

Hello JagelBagel,

Depending on the number of orders being sent at once you may be hitting a code limit, repetitive actions like submitting orders from loops can be blocked. Have you tried to remove the loop and submit the orders in that way? Does it still have a problem? Do you see a problem if you use less orders?

[JagelBagel]

Hello Jesse,

Thank you so much for sharing this idea... however, I don't know how I would avoid using a loop without entirely restructuring my code. The strategy is currently only placing maximum 2 trades per bar (and only about 100 trades were hit over 11000 bars). The strategy also only has limits open within a 100 point range (testing on NQ) so usually there are no more than 4 limit orders on each side at a time.
Do you still think that could be the issue and if so, is there a way to turn off the code limit?

[NinjaTrader_Jesse]

Hello JagelBagel,

You would have to try removing the loop to know if that is related to the problem or not.

I would otherwise be unsure of why an order may not be submitted, if you have a way of reproducing that result in a simple way we could explore that further.

[JagelBagel]

Ok thanks Jesse. I will work on that and get back to you. I appreciate you taking the time to help

[JagelBagel]

Hi Jesse,

Sadly it looks like it was not the loop. I tried using a bunch of if statements instead and I got the same result. Still entering internal SubmitOrderManaged() method, still equal to null right after setting the order. Here is a bit more of the code. Please let me know if you have any other ideas!

This is in the EvaluateLevels function that returns levelsToSubmit as an int list (referenced as masterSellEvaluated[1] in the previous code).

// Sell orders
if (levels[i].order == null && levels[i].lowPrice - (FrontrunTicks * TickSize) - Closes[BarsInProgress][0] <= OrderRange)
{
    // If the level is in range and has no order, submit the order
    if (levels[i].seriesNumber == -1)
        PrintDebug("hi Level in range and has no order: " + levels[i].signalName);
    levelsToSubmit.Add(i);
    continue;
}​

So I call this function and place the trades using a bunch of if statements now (I only included one, but there are many).

List<int>[] masterSellEvaluated = EvaluateLevels(masterSellLevels, false);
// Trying if statements
if (masterSellEvaluated[1].Count >= 1)
{
    int levelIndex = masterSellEvaluated[1][0];
    PrintDebug("This is the master list: " + masterSellLevels);
    PrintDebug("LEVEL IN MASTER LIST: " + masterSellLevels[levelIndex]);
    masterSellLevels[levelIndex].order = EnterShortLimit(0, true, Quantity, masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize), masterSellLevels[levelIndex].signalName);
    if (masterSellLevels[levelIndex].order == null)
    {
        PrintError("abc123 FAILED AT: " + (masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize)));
    }
    else
    {
        PrintInfo("ENTERED SHORT LIMIT AT: " + (masterSellLevels[levelIndex].lowPrice - (FrontrunTicks * TickSize)));
        PrintDebug("The level in the master list: name - " + masterSellLevels[levelIndex].signalName + " | order details - " + masterSellLevels[levelIndex].order + " , " + masterSellLevels[levelIndex].order.LimitPrice + " , " + masterSellLevels[levelIndex].order.OrderAction);
    }
}​

And here is the level class:

public class MyLevel
{
    public int barsAgo;
    public double highPrice;
    public double lowPrice;
    public bool isBuy;
    public double nearestSwing;
    public string signalName;
    public Order order;
    public int seriesNumber;
}​

Thanks again for looking at this. I will try to reproduce the issue with a simpler script when I have some time later this week. Cheers!

[NinjaTrader_Jesse]

Hello JagelBagel,

From the given code I don't see anything that specifically sticks out, the way you are assigning the order to a variable may or may not return a null value and that would be expected. Generally you need to use OnOrderUpdate to collect order variables once the order has been processed and at least one order event has happened.

[JagelBagel]

Hmm very strange that it would work with some and not with others then. Also, remember how the orders that are not working are not actually triggering any order events? I have log statements for each order state in OnOrderUpdate, but they are not printing anything even after the order trace says the order is being submitted. And if an order is equal to null, my strategy keeps attempting to place it on subsequent bars until it is invalidated, but the same orders do the same thing bar after bar.

Anyways, if no more ideas come to mind, I'll update this thread with a new script to reproduce the issue once I build it. Either that or hopefully in the process of making said script I will find my answer and post it. Thanks for trying, Jesse!

[NinjaTrader_Jesse]

Hello JagelBagel,

The order not triggering any events would mean that the order was not actually submitted, the order events need to happen to confirm the order was successfully submitted and begins working. The return of the method is just a carried over concept from NT7 but is not the suggested way to confirm order's or collect variables.

If you are able to produce a small sample that demonstrates the Entered internal SubmitOrderManaged() print you had shown and then no events from OnOrderUpdate that would be something I could look further into or report if that ends up being a bug.

[JagelBagel]

Hello Jesse,

I was able to make a little example script that follows a similar logic and produces the same result. Here is the strategy below.

#region Using declarations
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Input;
using System.Windows.Media;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Gui;
using NinjaTrader.Gui.Chart;
using NinjaTrader.Gui.SuperDom;
using NinjaTrader.Gui.Tools;
using NinjaTrader.Data;
using NinjaTrader.NinjaScript;
using NinjaTrader.Core.FloatingPoint;
using NinjaTrader.NinjaScript.Indicators;
using NinjaTrader.NinjaScript.DrawingTools;
using NinjaTrader.NinjaScript.MarketAnalyzerColumns;
#endregion

//This namespace holds Strategies in this folder and is required. Do not change it.
namespace NinjaTrader.NinjaScript.Strategies
{
    public class TestLevel5328
    {
        public double price;
        public bool isBuy;
        public string signalName;
        public Order order;
        public int seriesNumber;
    }

    public class OrderTesting : Strategy
    {
        private EMA baseline0;
        private EMA baseline1;
        private List<TestLevel5328> buyLevels = new List<TestLevel5328>();
        private List<TestLevel5328> sellLevels = new List<TestLevel5328>();
        private double orderRange = 100.00;
        private int offsetTicks = 10;

        protected override void OnStateChange()
        {
            if (State == State.SetDefaults)
            {
                Description                                    = @"Testing orders not placing oct 2024";
                Name                                        = "OrderTesting";
                Calculate                                    = Calculate.OnBarClose;
                EntriesPerDirection                            = 1000;
                EntryHandling                                = EntryHandling.AllEntries;
                IsExitOnSessionCloseStrategy                = true;
                ExitOnSessionCloseSeconds                    = 30;
                IsFillLimitOnTouch                            = false;
                MaximumBarsLookBack                            = MaximumBarsLookBack.TwoHundredFiftySix;
                OrderFillResolution                            = OrderFillResolution.Standard;
                Slippage                                    = 0;
                StartBehavior                                = StartBehavior.WaitUntilFlat;
                TimeInForce                                    = TimeInForce.Gtc;
                TraceOrders                                    = true;
                RealtimeErrorHandling                        = RealtimeErrorHandling.StopCancelClose;
                StopTargetHandling                            = StopTargetHandling.PerEntryExecution;
                BarsRequiredToTrade                            = 20;
                // Disable this property for performance gains in Strategy Analyzer optimizations
                // See the Help Guide for additional information
                IsInstantiatedOnEachOptimizationIteration    = true;
            }
            else if (State == State.Configure)
            {
                AddVolumetric("MNQ 12-24", BarsPeriodType.Range, 40, VolumetricDeltaType.BidAsk, 1, 0, null, null);
            }
            else if(State == State.DataLoaded)
            {
                baseline0 = EMA(Closes[0], 20);
                baseline1 = EMA(Closes[1], 20);
            }
        }

        protected override void OnExecutionUpdate(Cbi.Execution execution, string executionId, double price, int quantity,
            Cbi.MarketPosition marketPosition, string orderId, DateTime time)
        {
            
        }

        protected override void OnOrderUpdate(Cbi.Order order, double limitPrice, double stopPrice,
            int quantity, int filled, double averageFillPrice,
            Cbi.OrderState orderState, DateTime time, Cbi.ErrorCode error, string comment)
        {
            if (order.OrderState == OrderState.Cancelled)
            {
                Print("Order cancelled: " + order.Name);
            }
            else if (order.OrderState == OrderState.Rejected)
            {
                Print("Order rejected: " + order.Name);
            }
            else if (order.OrderState == OrderState.Unknown)
            {
                Print("Order unknown: " + order.Name);
            }
            else if (order.OrderState == OrderState.Suspended)
            {
                Print("Order suspended: " + order.Name);
            }
            else if (order.OrderState == OrderState.Filled)
            {
                Print("Order filled: " + order.Name);
            }
            else if (order.OrderState == OrderState.Accepted)
            {
                Print("Order accepted: " + order.Name);
            }
            else if (order.OrderState == OrderState.Working)
            {
                Print("Order working: " + order.Name);
            }
            else if (order.OrderState == OrderState.PartFilled)
            {
                Print("Order part filled: " + order.Name);
            }
            else if (order.OrderState == OrderState.TriggerPending)
            {
                Print("Order trigger pending: " + order.Name);
            }
            else if (order.OrderState == OrderState.Submitted)
            {
                Print("Order submitted: " + order.Name);
            }
            else if (order.OrderState == OrderState.CancelPending)
            {
                Print("Order cancel pending: " + order.Name);
            }
            else if (order.OrderState == OrderState.ChangePending)
            {
                Print("Order change pending: " + order.Name);
            }
            else
            {
                Print("Order state: " + order.OrderState + " " + order.Name);
            }
        }

        protected override void OnBarUpdate()
        {
            //Add your custom strategy logic here.
            if (CurrentBars[0] < BarsRequiredToTrade || CurrentBars[1] < BarsRequiredToTrade)
            {
                return;
            }

            if (BarsInProgress == 0)
            {
                // Check for new levels
                if ((Closes[0][0] > baseline0[0] && Closes[0][1] <= baseline0[1]) && (Closes[1][0] > baseline1[0]))  // Long signal
                {
                    double levelPrice = Lows[0][0] - (offsetTicks * TickSize);
                    buyLevels.Add(new TestLevel5328
                    {
                        price = levelPrice,
                        isBuy = true,
                        signalName = "LongSignal" + levelPrice,
                        seriesNumber = 0
                    });
                }
                else if ((Closes[0][0] < baseline0[0] && Closes[0][1] >= baseline0[1]) && (Closes[1][0] < baseline1[0]))  // Short signal
                {
                    double levelPrice = Highs[0][0] + (offsetTicks * TickSize);
                    sellLevels.Add(new TestLevel5328
                    {
                        price = levelPrice,
                        isBuy = false,
                        signalName = "ShortSignal" + levelPrice,
                        seriesNumber = 0
                    });
                }

                int qty = 1;
                List<int>[] buyLevelsToChange = EvaluateLevels(buyLevels);
                foreach(int levelIndex in buyLevelsToChange[0])
                {
                    // Submit orders
                    buyLevels[levelIndex].order = EnterLongLimit(0, true, qty, buyLevels[levelIndex].price, buyLevels[levelIndex].signalName);
                    if (buyLevels[levelIndex].order != null)
                    {
                        PrintBar("Order submitted: " + buyLevels[levelIndex].signalName);
                    }
                    else
                    {
                        PrintBar("Order not submitted: " + buyLevels[levelIndex].signalName);
                    }
                }
                foreach (int levelIndex in buyLevelsToChange[1])
                {
                    // Cancel orders
                    CancelOrder(buyLevels[levelIndex].order);
                    buyLevels[levelIndex].order = null;
                }
                for(int i = 0; i < buyLevelsToChange[2].Count; i++)
                {
                    // Remove levels
                    int levelIndex = buyLevelsToChange[2][i] - i;
                    buyLevels.RemoveAt(levelIndex);
                }

                List<int>[] sellLevelsToChange = EvaluateLevels(sellLevels);
                foreach (int levelIndex in sellLevelsToChange[0])
                {
                    // Submit orders
                    sellLevels[levelIndex].order = EnterShortLimit(0, true, qty, sellLevels[levelIndex].price, sellLevels[levelIndex].signalName);
                    if (sellLevels[levelIndex].order != null)
                    {
                        PrintBar("Order submitted: " + sellLevels[levelIndex].signalName);
                    }
                    else
                    {
                        PrintBar("Order not submitted: " + sellLevels[levelIndex].signalName);
                    }
                }
                foreach (int levelIndex in sellLevelsToChange[1])
                {
                    // Cancel orders
                    CancelOrder(sellLevels[levelIndex].order);
                    sellLevels[levelIndex].order = null;
                }
                for (int i = 0; i < sellLevelsToChange[2].Count; i++)
                {
                    // Remove levels
                    int levelIndex = sellLevelsToChange[2][i] - i;
                    sellLevels.RemoveAt(levelIndex);
                }
            }
        }

        private List<int>[] EvaluateLevels(List<TestLevel5328> levelsList)
        {
            List<int> levelsToSubmit = new List<int>(); //0
            List<int> levelsToCancel = new List<int>(); //1
            List<int> levelsToRemove = new List<int>(); //2
            for(int i = 0; i < levelsList.Count; i++)
            {
                // Submit in range
                if (levelsList[i].order == null && levelsList[i].price >= Closes[BarsInProgress][0] - orderRange && levelsList[i].price <= Closes[BarsInProgress][0] + orderRange)
                {
                    levelsToSubmit.Add(i);
                    PrintBar("Added level to submit: " + levelsList[i].signalName);
                    continue;
                }
                // Cancel out of range
                if (levelsList[i].order != null && (levelsList[i].price < Closes[BarsInProgress][0] - orderRange || levelsList[i].price > Closes[BarsInProgress][0] + orderRange))
                {
                    levelsToCancel.Add(i);
                    PrintBar("Added level to cancel: " + levelsList[i].signalName);
                    continue;
                }
                // Remove levels that are hit
                if ((levelsList[i].isBuy && Lows[BarsInProgress][0] <= levelsList[i].price) || (!levelsList[i].isBuy && Highs[BarsInProgress][0] >= levelsList[i].price))
                {
                    levelsToRemove.Add(i);
                    PrintBar("Added level to remove: " + levelsList[i].signalName);
                    continue;
                }
            }
            return new List<int>[] { levelsToSubmit, levelsToCancel, levelsToRemove };
        }

        private void PrintBar(string msg)
        {
            Print("[" + BarsInProgress + "]-[" + (CurrentBar) + "] " + msg);
        }
    }
}
​

Also thank you for the information on the method return and value setting. I will make another variation using a different method to set level.order by matching signal name to order name. Do you think that would work better?

[NinjaTrader_Jesse]

Hello JagelBagel,

Thanks for providing a sample. I will try this, are you able to see this happen while using the Sim101 account as well?

Regarding using the other approach, that would be suggested to correctly assign the variables after they are successfully submitted. The OnOrderUpdate is only called once the order is successful and an order object is returned.

[JagelBagel]

I'm live testing the OrderTesting script right now as the market closes. I am noticing some strange behavior with orders on the opposite side of the market all cancelling once an order is filled. Not sure if that is happening in my other strategy or if that is part of the issue, but do you know how I would fix that while we are here?
So far I have not noticed any entries acting up, but I will keep an eye out.

Thanks again for your help and have a great weekend

[NinjaTrader_Jesse]

Hello JagelBagel,

I would not be sure about that, that may relate to the signal names being used. To explore that further it would be best to remove the loops and other complexities you have to get down to a working state. In that working state if you still see the problem you originally reported we would be able to explore that seeing exactly what actions are being done without having to use prints to explore the logic as it is looping.

[JagelBagel]

Hello Jesse,

I can confirm I am seeing the same behavior (order not submitting) while live testing on the demo account. I will rewrite the entire strategy using a different approach when I get time. Are there any best-practice resources for limit order management that you would be able to point me to that you think would work for my situation?

[NinjaTrader_Jesse]

Hello JagelBagel,

Aside from not using loops to where you may hit a action limit there are not any specific best practices I could suggest. Using limit orders from NinjaScript requires that you call the method to submit the order. To get order variables you can use OnOrderUpdate to both confirm the submission was successful and get the variable.