Exits matching up with incorrect entries despite providing correct EntrySignalNames.

[ChiTrader2000]

Hello,

I have been blown away with what an awesome C#-tweakable backtesting engine Ninja Trader is and am EXTREMELY close to creating a custom application with it, but I believe that I have found an actual Ninja Trader bug that is keeping my application from being usable.

Here is the problem (image of chart and output window are attached. Chart makes the problem very easy to understand.):

1. I enter a Long position with order named "Stop LE 30".

2. I later enter another Long position with order named "Stop LE 31".

3. I later ATTEMPT to exit the second order ("Stop LE 31") with an order called "X_Stop LE 31", BUT it bizarrely exits "Stop LE 30".

4. I later ATTEMPT to exit the first order ("Stop LE 30") with an order called "X_Stop LE 30", BUT it bizarrely exits "Stop LE 31".

I am using the proper entry signal names each time, as can be seen by the image of the attached Output log, which shows that I am using the proper Entry Signal Names even according to the internal Ninja Trader engine.

I have attached an image of a chart with two entries and two exits, and an image of the corresponding rows that were printed to the Output Window from both the internal NinjaTrader TraceOrders feature, as well as rows from my Print statements before and after I call each Enter and Exit function.

Notice how the chart drew the connecting lines between the wrong entries and exits, and notice how the Ninja Trader's TraceLog shows that the internal Order Handling of Ninja Trader appears to be ATTEMPTING to PROPERLY match up the entries and exits, yet the entries and exits on the Chart (and on the (not attached) Trades performance grid) are matched up incorrectly.

A few details...

1. I have three time frames on the chart: 1) 1 minute, 2) Daily, and 3) 1 minute again.

2. Entries are written like:
EnterLong(2, 1, entryName);

3. Exits are written like:
this.ExitLong(2, 1, "X_" + entryOrder.Name, entryOrder.Name);

Thank you to ANYBODY who can help me out here!

ChiTrader2000

[NinjaTrader_Bertrand]

Hi ChiTrader2000,
Thanks for your post and feedback - please post more code and the log file if you don't mind, this makes it easier to comment on the behavior you see. Thanks!

[ChiTrader2000]

Supporting Code, Log File, Output Window, and Data for Entry-Exit Mismatch Problem

Hello,

THANK YOU SO MUCH FOR LOOKING INTO THIS!

I have attached a zip file with the following contents:

1. All code: Includes "SubsystemBase.cs", which contains the base class and supporting classes, and "SubsystemOneTB.cs", which contains the SubsystemOneTB strategy, which inherits from SubsystemBase, and is the strategy that was displaying the problems.

2. Log File CSV Export: Note that ALL the log file entries are on 12/16/2008 or before, but I just ran the Strategy today (12/17/2008), the strategy still showed the problem, and the Log File didn't display any problems.

3. Output Window: The problems start with the TraceOrders printed rows for trade date 10/10/2008, as in the image in my last post.

4. Daily ES 12-08 data from 9/22-10/13.

5. 1 Minute Bar ES 12-08 data from 10/9 - 10/10.

The back test was for the SubsystemOneTB strategy on the ES 12-08 contract using the data I have included. There is lots more daily and 1 minute bar data before and after these dates, but I believe that I have given you enough data for the back test to work.

Please post back if you need ANYTHING else to help debug this problem.

THANK AGAIN!

ChiTrader2000

[NinjaTrader_JoshP]

ChiTrader2000,

Unfortunately your code is stepping outside the scope of what we can offer support for. You are using more advanced C# that we just do not have the bandwidth to cover. What I suggest you try is an as-simple-as-possible strategy and see if you experience the same issue.

[ChiTrader2000]

Simple code that shows the Ninja Trader Entry-Exit Mismatch Bug

Hello NinjaTrader_Josh (or ANYBODY who can help),

Your last post made the excellent suggestion to, "try is an as-simple-as-possible strategy and see if you experience the same issue."

I took your advice with a VERY simple sample and experienced the exact same issue. As far as I can tell, I have uncovered a critical bug in Ninja Trader that is completely in the best interests of Ninja Trader to fix, or to at least publish a workaround.

The attached images "EntryExitMismatchBug_Chart.GIF" and "EntryExitMismatchBug_Trades_Grid.GIF" shows the bug where Ninja Trader mismatches two simple exits with two simple entries, and the attached C# file "EntryExitMismatchBug.cs" is the very simple Ninja Trader strategy that causes this behavior.

Note how the strategy explicitly attempts to exit "Entry1" with "Exit1" and "Entry2" with "Exit2" with the following code:

...
this.ExitShortStop(0, true, 1, 1155.00, "Exit1", "Entry1");
...
this.ExitShortStop(0, true, 1, 1135.00, "Exit2", "Entry2");
...

yet the chart and the Trades Grid shows that Ninja Trader is exiting "Entry1" with "Exit2" and "Entry2" with "Exit1".

To reproduce this bug, you will probably need to import the e-mini S&P 1 minute bar data for 9/26/2008 to 9/30/2008 from the file I zipped up in ES 12-08.zip, and also you will probably need to set your backtest properties dialog box settings as shown in the attached image called "EntryExitMismatchBug_BackTest_Settings.GIF" (Note how the back test is for the 1 Minute Bar data series, the Session Begins and Ends are 8:30 AM-3:15 PM, Exclude Weekend=True, the From-To=9/29/08-9/29/08, Fill type = Liberal, Entries per direction=100000, and Entry handling=AllEntries).

Please respond to this post, I beg of you. My particular usage of Ninja Trader will blow my users' minds if this bug is fixed or a workaround is published, but will be useless to them without a fix. I will do anything you need to help find a solution or workaround to this Ninja Trader problem. Please post ANY request for further information and I will get back to you ASAP.

Speaking from experience, Ninja Trader is a MUCH better backtesting application than some very famous major competitors with much bigger programming and advertising budgets. This bug is the only major one that is keeping my company from using it for lots of projects.

Thank you in advance to ANYBODY who can solve this problem!

ChiTrader2000

[NinjaTrader_JoshP]

ChiTrader2000,

After reviewing your code it is evident that internal signal tracking has been screwed up. I did not investigate as to the exact step that it has gone awry at, but please see the attached changes I have made. The orders you were submitting shouldn't have even been valid orders and they were not picked up. Please see the modifications I have made.

[ChiTrader2000]

More important information based on your last post.

Hi NinjaTrader_Josh,

First off, thank you for your time and effort in looking at this problem, and thank you for looking close enough to notice that there is in fact an actual bug in Ninja Trader (among a million features that work perfectly).

Is this something that you guys can look at fixing soon?

Second, I tried the strategy you posted, and instead of fixing the mismatch problem, it didn't exit at all (until it exited on close). I have attached an image ("EntryExitMismatchBug_Chart_JoshsChgs1.Gif") of the chart showing that the Exits from your strategy are not working.

Third, can you explain to me why the orders in my strategy shouldn't have been valid?

Fourth, you said that my orders were not "picked up", but please view the attached Output window image ("EntryExitMismatchBug_TraceOrdersTrue.gif") to see how my two entry and two exit orders were nicely picked up by Ninja Trader, at least according to the order tracing that Ninja Trader does when "TraceOrders = true;". Is it possible that we are running different version of NT? Maybe the orders aren't getting picked up in your version but they seem to be getting picked up in my version (my version is v. 6.5.1000.7).

Thanks again for your time and consideration of the bug that I have posted.

Looking forward to your response.

ChiTrader2000

[NinjaTrader_JoshP]

1. The issue is caused by misuse of code.

2. That is the point. It is not suppose to exit because your exit orders are invalid. In your original code your exit orders were illegal orders, but somehow your coding style has messed up the internal order logic. The strategy I posted restores the internal order logic and that is why your illegal orders were blocked. Please see the TraceOrders logs for more detail.

3. The price you submitted at are invalid for those points in time. Please see the TraceOrders output for the exact error message regarding your orders.

4. By picked up I mean going through the internal order logic. It is evident your orders were not going through that logic because they were not rejected. Please see the error messages you were suppose to get through TraceOrders on the prior attached strategy.

[ChiTrader2000]

Further details and new ideas based on your last post.

Hi NinjaTrader_Josh,

1. After a lot of experimentation, I now see what you are talking about regarding illegal ExitShortStop prices. Thanks for the explanation.

Both orders from your strategy were ignored for the same reason that both of mine were not ignored; because the Close Price of the bar that you placed your short exit stops on was higher than the Short Exit Stop Prices, which as you mentioned is illegal, whereas by delaying placing my Short Exit Stop for one bar in a market that was moving straight down but using the same absolute prices for my Short Exit Stop Prices, I was able to place the orders because the Close Price of the bar I placed my short exit stops was at this point lower than the Short Exit Stop Prices, which is legal. So it makes sense that your orders were rejected and mine were accepted.

2. You said that my coding style has messed up the internal order logic. I did not intend to break the Ninja Trader engine with my simple and straightforward code that submitted legal orders. I was simply trying to achieve the very desirable behavior that can be seen in the attached image "EntryExitMismatchBug_Desired_Behavior.gif". Notice how nicely the "tight short exit stop" gets placed (not ignored) then executed at a bar in the future that breaches the stop. The image shows it all. The code that produced this image is attached. I merely commented out the second entry and exit lines, to avoid the entry-exit mismatch problem. Doing this works like a charm, so I KNOW that NT can handle this "wait one bar before placing the short exit stop" code very nicely, although the current version can't handle it when the time ranges of multiple entries and exits overlap. You guys are so close to a perfect product. Perhaps one small fix could fix the entry-exit mismatch behavior for overlapping positions. This wouldn't just be great for me, but great for you guys at Ninja Trader as well.

Could someone at NT at least take look at this specific "entry-exit mismatch when positions overlap" problem that I have detailed, for both of our benefits?

Thanks in advance for considering my request.

ChiTrader2000

[NinjaTrader_JoshP]

ChiTrader2000,

From what I saw yesterday your orders are invalid. You submit at a price higher than the market. The data is the same and does not change the fact that the orders are submitted at illegal prices at that given moment in time. The strategy I put together does the exact same thing you have done.

Unfortunately I cannot spend time to help you debug your case scenario. Please just follow the reference I put together based off of your strategy. The strategy was left "unoperational" to show you the problem with the orders. If you submitted at valid prices you should find the orders close out properly.

Please take a look at the attached screenshot. You submit your order at 1155. That price is nowhere near valid anywhere on the bar the order was sent in at. This order should have been rejected as shown by the reference I attached earlier. Whatever you are doing in your code, whether it is the function you made and race conditions or the "this". I cannot help you determine the problem area since all of those are outside the scope of what we can support. We can support NinjaScript, but not C#. On my end, everything works as expected from standard NinjaScript.

[ChiTrader2000]

NinjaTrader_Josh

Thank you for your quick response.

1. You said that "From what I saw yesterday your orders are invalid. You submit at a price higher than the market."... My ExitShortStop orders were valid, BECAUSE I submitted at price higher than the market, right? I thought that ExitShortStop orders NEED to be placed above the market, and that's what I have been doing this entire time. Am I totally wrong here about how one needs to place ExitShortStop orders ABOVE the market?

2. The screenshot you attached proves my point above perfectly. You yourself are correctly showing that I am placing my ExitShortStop order at 1155, far above where the market is at, making the ExitShortStop legal, right?

3. You said that "The strategy I put together does the exact same thing you have done.", but your strategy places the orders one bar earlier, when the ExitShortStop orders ARE illegal because the submission bar's Close Price was above the ExitShortStop order price. That's why your ExitShortStop orders were rejected (per the Output window) and mine were accepted. If the below statements are true, then NT's rejection of your strategy's orders does NOT show that the entry-exit mismatch problem is due to my orders... your orders were illegal, which is why they were rejected... mine were legal so they were accepted. Now you understand why I waited one bar before I placed my orders... to make sure that they were LEGAL, as my original output window image (reattached again as EntryExitMismatchBug_TraceOrdersTrue.gif"), and the fact that NT accepted these orders, shows.

Am I wrong that ExitShortStop orders need to be placed at prices above the market?

If not, why would my simple, straightforward code that waits one bar after entry to place the ExitShortStop orders (ABOVE the market) be the reason for the NT "entry-exit mismatch when positions overlap" bug, when, per your own diagram, NT nicely fills my order and matches the entry and exit correctly when I comment out the entry and exit for the second overlapping order?

Are my orders really the problem, or is there a small glitch in Ninja Trader that really should be in you guys' best interest to take a real look at?

Thanks,

ChiTrader2000

[NinjaTrader_JoshP]

ChiTrader2000,

I finally see what you are saying. Rest assured we are looking at it.

[ChiTrader2000]

NinjaTrader_Josh,

Thank you so much!

Once entries and exits match up, even for overlapping positions, I will have got past the only really critical issue that is left for me to produce a finished Ninja Trader application that is so far beyond what I was able to do with another expensive, overhyped "major competitor's product" that my many users (all top traders that know lots of other traders) will be blown away. They need to be able to do lots of scaling in and out and they will be scrutinizing the entry and exit prices very closely, which is why it is so important that the entries and exits match up even for overlapping positions. They will reject my application if they don't believe that I am always giving them fair fills for their exits, based on their entries, but if they believe in the fairness of the fill price difference between the entries and exits, they will be glued to the Ninja Trader Strategy Analyzer, trying to iteratively improve their trading ideas.

I plan to create a couple dozen influential diehard Ninja Trader fans with my finished application.

Looking very much forward to what you guys find out! I will keep checking this forum thread for an update!

Thank you so much once again for looking into this!

Happy New Year!

ChiTrader2000

[zark.be]

i'm experiencing a similar problem.

BUT running the strategy "live" actually works for mine.

In the strategy analyzer, it's doing some sort of "First In, First Out"

and in the "live" chart, it's linking them correctly !

ChiTrader2000> try the same thing, maybe it'll help us find the solution

[ChiTrader2000]

zark.be,

Thank you for posting this important new information. I am sure it will be of interest to the Ninja Trader guys who are looking into this problem.

I don't currently have access to my company's live data streams so I can't try your suggestion at the moment, but lucky for us, they are now looking into this problem, which per your new information, may only be occurring in backtesting mode.

Let's see what they find out and hope for the best!

ChiTrader2000