Historic to real time trade transitioning

[traderpards]

I'm trying to deal with the error message that precedes shutting down the strategy that goes something like "Transitioning order references from historical to live" and it directs you to check the help guide.

I found this code in the help guide, or something like it and implemented it.

    else if (State == State.Realtime)
    {
           // one time only, as we transition from historical convert any old historical order object 
           // references to the new live order submitted to the real-time account
           if( entryOrder != null ) 
           {
                entryOrder = GetRealtimeOrder( entryOrder );
                if( entryOrder != null )
                {
                    Print( entryOrder.ToString() );
                }
                else {
                    Print( "entryOrder is null" );
                }
           }
    }

Is this all I need to do? Because every time I do, the method GetRealtimeOrder() seems to always return null. I put my strategy on last night a little late because I was debugging something and a historical trade had already triggered. (On the screen shot the strategy was launched in-between those two trades. Start behavior = Wait until flat) Then a real-time trade triggered. (I think because it produced a profit for one contract - please see the screen shot). But then one of them, it looks like the real time trade generated a trade in the opposite direction on exit and it has no stop or target, which is obviously not good. But was the real time trade really real? Those never were in NT7 but the control center is reporting a realized profit for it.

Since the doc (*Transitioning order references from historical to live) says GetRealtimeOrder() should return a real time order reference, I'm thinking at the very least, that method isn't working right, in addition to something either I'm not doing or don't understand.

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your post.

I am unable to replicate this same behavior using my own strategy based on your provided code, details, screenshot, and settings. Do you have a strategy that produces this behavior that we may test?

Can you send us your log and trace files for this occurrence? You can do this by going to Help > Email Support. Please list 'ATTN: Patrick H' in the subject line and reference this thread in the body of the email.

[traderpards]

Sorry! I never saw your reply and was actually wondering about this issue when I got "entryOrder is null" in one of my strategies this evening and that's when I found out you had replied.

OK, I'll try to include exact steps to show you what I'm doing, because something isn't right.

1.) Import the attached strategy. It's just a moving average cross with low settings to get it produce a historical trade without having to wait too long.
2.) Create a chart with a tight time frame - say 4 Range with an instrument whose moving averages meanders a lot, like GC or ES.
3.) Open an Output window.
4.) Start the strategy - MACrossRealTimeTransitioningTest.
5.) If you're lucky on the first try, you'll get a historical trade and you'll see "entryOrder is null" in the Output Window. If you didn't get a historical trade, keep playing with the averages or the bar period until you do. You should get one before too long and you'll see that method returns null, every time. The idea is to get a chart with a historical trade so you can see how it's transitioning. I don't think it's doing it right.
6.) Now for the part that's iffy. I'm not sure if in my chart I attached in my original message if that trade I have labeled is really Real Time or not. I think it is but I have all my defaults set to Wait Until Flat so it doesn't make sense that it is, particularly since I never mess with that setting. But somehow it triggered a real trade in the opposite direction, when it hit the profit target, which was definitely in real time. And either way, it shouldn't have done that. I don't know how it triggered it but I'd see if I could wait until the historical trade hits a profit target so you can find out if this does too. I think the best plan of attack is to try to get into a situation where you have two entries and they both hit profit targets, instead of flipping around in the other direction. If it does flip around, try restarting the strategy so you have a historical trade again and one of these times the historical trade will hit the profit target and see if it does to you what it did to me; that is, enter an unwanted trade.

You may not be able to get it to make that renegade entry. But at the very least I'd love to know why GetRealTimeOrder() is supposed to return null, if you don't mind. Then I can at least exonerate GetRealTimeOrder() and try to figure out why I got an unwanted trade in the opposite direction.

Thanks!

[traderpards]

Hmmm... It looks like the Zip file didn't attach. Sorry about that! That's OK, I have more on this...

It turns out that maybe I did remember correctly - that second trade really was a real time trade, even though I have Wait Until Flat selected. I'm confident because this test did the same thing. Notice in the screen shot where I refreshed the screen to get a historical trade. Then a live trade triggered and the framework accepted it. Then, the MAs crossed and we triggered a reversal. But the framework thought two live trades were active and it closed two live trades when it should have closed one and that resulted in going short two in the opposite direction when I should be short one. There it is!

First screen shot: I missed it so there is no screen shot attached. I was going so fast I guess I forget to set the chart focus before I hit Alt-Print Screen. It showed two short with stop and target of one each. I guess you'll have to take my word for it.

Note: First screen shot isn't attached because I didn't set the focus on the chart properly and missed it. But the next four are.

Second screen shot I typed up a bunch of text to show the historical trade, the live trade that seems to me should never have been taken, and the reversal. But I didn't get until that reversal trade hit its profit target. But you can clearly see that I still have a live orphaned trade with no stop or target.

Third screen shot: This shows I'm flat but I'm not. You can see a trade there around 8:10.

Fourth screen shot: It almost looks like we're right again but we're not. Chart Trader says I'm flat but I really long one.

Fifth: I think we're screwed. It looks like we'll always be off from here on out, all because either me or the framework didn't handle that historical trade properly. I'm actually two long here even though Chart Trader shows one.

But I did what the documentation told me... I must be missing something.

[traderpards]

The strategy attached in the previous message is the one I meant to attach last night. But after I thought I attached that, I played with it some more to try to make it better as far as keeping track of the entry orders a little better. I'm not sure I succeeded but attached is the one I ran all night and is the one in the screen shot.

Now you absolutely have everything you need to reproduce this, like I've reproduced it.

[traderpards]

More

So I had to stop my strategy because of a historical trade that ended up going in the wrong direction. When I restarted it, here is what it looked like (attached)

So I was thinking, maybe there is nothing wrong with GetRealtimeTrade() returning null, (except the docs don't say that's what is supposed to happen) maybe my problem is the stops and targets are what is going live. I thought, "I'll be you a dollar that thing goes in the opposite direction when it hits the stop or the target.

Well I didn't have to wait long to get my answer (second attachment) Bam! It hit the stop and now I'm three long. Except I should be two long because one of the targets on an Entry-3 was already hit. But that's beside the point. The point is, it shouldn't have even gone long in the first place, quantity aside. (When targets are hit, I change the quantity in OnExecutionUpdate() so if the historical engine doesn't do something in there, that's OK.)

The thing is, how do I fix that? It seems to me, I can't on my end. It seems to me, the framework shouldn't be taking historical stops and losses in the opposite direction and I can see how maybe some flag never got set in those.

What's different this time is Chart Trader made it possible to see those stops and targets and it usually doesn't. That's a historical trade in the first screen shot so you'd think you wouldn't be able to see those.

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for the additional information.

I am still testing this to try to reproduce and isolate the cause. Once I have additional information I will follow up here.

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your patience.

I would like to test your configuration for the NinjaTrader 8 on my end. Can you send over your configuration file? You will find your configuration file in the following director:

• Documents\NinjaTrader 8\config.xml

Please attach this file to an email to platformsupport[at]ninjatrader[dot]com with 'ATTN: Patrick H' in the subject line and a reference to this thread in the body of the email.

I look forward to assisting you further.

[traderpards]

I would like to test your configuration for the NinjaTrader 8 on my end. Can you send over your configuration file? You will find your configuration file in the following director:

• Documents\NinjaTrader 8\config.xml

It's on its way...

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your patience.

I am able to reproduce with your settings. I will update this thread when I have further details.

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your patience.

The exit orders are transitioning into realtime orders, therefore they are executing. So while you have the strategy set to wait until it's position is flat, the exit orders it has submitted are becoming active in realtime and when they close the strategy position they are really taking positions.

To correct this you would not want the exit order objects to transition to realtime when you are running WaitUntilFlat.

Please let me know if you have any questions.

[traderpards]

Your suggestions didn't work

Good morning,

The documentation is either wrong or incomplete. I just got the same message I was asking about and I did what you suggested; that is, only call GetRealtimeOrder() on only the entries:

            else if (State == State.Realtime)
            {
                // One time only, as we transition from historical convert any old historical order 
                // object references to the new live order submitted to the real-time account
                foreach( TradeStruct m in allTrades )
                {
                    // We only want to make this call on the entries because while transitioning into a 
                    // realtime order it turns those stops and targets into live trades.
                    if( m.Entry != null ) {
                         m.Entry = GetRealtimeOrder( m.Entry );
                    }
                }
            }

The comment I made above reflected what you told me to do. However, in this case I did have to stop and restart the strategy and up popped up that targets and stops, even though I did not call GetRealtimeOrder() on the stops and targets. So when the price hit a target, the error was generated.

I know you guys have an aversion to actually running the program but in this case you probably will have to. Will you please so you can see what I'm talking about?

1. Nothing works like the documentation says. I see that you updated the documentation with: "If no associated order exists (i.e. OrderState is Filled, Canceled, Rejected, Unknown), a null value returns." I can assure you that a historical order was indeed present. I don't know if null was returned because frankly, I didn't expect to get the same thing that I got when I first raised this issue. And when I've stepped through this in debug mode, I've never gotten anything BUT null values returned when historical orders were definitely present. (The error message even tells you that a historical order was present - the framework knows it but doesn't handle it properly.)
2. Historical orders should NOT generate stops and targets, but they sometimes do. I can confidently say this because I saw with my own two eyes the profit targets and stop losses were present on a historical order.

This is an intermittent issue. Since you told me about this and I made the code changes, I haven't had a problem until today. You will have to run your program in order to see it. I can't give you "exact steps" to reproduce the problem.

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your response.

I see, I overlooked the fact that if we adjust the exit orders in realtime it would still generate that message. In the case we have exit orders that would be managed but we use WaitUntilFlat we then need to implement logic to avoid adjustment of exits in State.Realtime until the first entry order has filled in State.Realtime.

You could perform this with a check for State == State.Realtime. For example:

		protected override void OnOrderUpdate(Order order, double limitPrice, double stopPrice, int quantity, int filled, double averageFillPrice, OrderState orderState, DateTime time, ErrorCode error, string nativeError)
		{
			if (order.Name == "myEntry")
			{
				myEntryOrder = order;
				if (State == State.Realtime)
					IsFirstRealOrder = true;
			}
			
			if (order.Name == "myExit")
			{
				if (State == State.Realtime && !IsFirstRealOrder)
					return;
				myExitOrder = order;
			}
		}

This way you would not need to transition any exit orders with GetRealtimeOrder().

Please let me know if you have any questions.

[traderpards]

Thanks for your last post. I'm not sure I really undersood what you were trying to say but here's something else... Please see the attachment.

These are all historical trades - I had to restart the strategy this morning for some reason. When I restarted it, that trade whose profit target was hit hadn't hit its profit target yet and those target and stop flags weren't there. When it did, those flags popped up. Why did it do that? Those are historical trades and we shouldn't have had that happen - those flags, it seems to me should never have popped up. Isn't that a problem?

That's the only question I have right now. It seems to me it's a problem because when the price hits them, it will result in a real-time trade. What do you say? (Because all the fiddling with the code won't help if you don't think it's a problem.)

[NinjaTrader_PatrickH]

Hello traderpards,

Thank you for your response.

You would be correct that the orders would result in real-time positions. In this test what Start Behavior were you using? Are you transitioning historical orders to real-time still?

I look forward to your response.