Market Replay Backtesting

[ScoobyStoo]

Hi folks,

I'm currently running NT v7.0.0.10 and am thinking about using it for some backtesting using market replay data. I tried to do this in NT v6.5 but NT's internal order management logic would not allow my strategy to operate as required...so I decided to wait for NT7.

I need to completely switch off all of NT's internal order management logic such that NT simply processes the order generation, modification and cancellation signals generated by my custom strategy code. I can't find any Help Guide documentation for the NT7 API (if any exists please tell me) so I've just been looking through the object model to see what new methods are available.

Can I confirm what steps I need to take to turn off all of NinjaTrader's order management logic such that my order signals are blindly processed by the market replay engine without any interference.

As far as I can tell the steps are:

1. Set StrategyBase.Unmanaged = true
2. Set StrategyBase.IgnoreOverFill = true (I am confused by the description for this property, how can an order be 'overfilled'?)

I should then use only the following methods to place, modify and cancel orders:

StrategyBase.SubmitOrder()

Since there are no overloads for this method I'm guessing that I'm supposed to just set the parameters which aren't applicable to a particular order type to zero (e.g. limit price for market orders)?

StrategyBase.ChangeOrder()

As above, since there are no overloads for this method I'm guessing that I'm supposed to just set the parameters which aren't applicable to a particular order type to zero (e.g. limit price for stop market orders)?

StrategyBase.CancelOrder()

This description for this method states that it is used for cancelling managed orders. Is this incorrect? I can see no other method to use for cancelling unmanaged orders.


Am I missing any other steps here?

Many thanks

[NinjaTrader_Bertrand]

Hi ScoobyStoo, this is correct using SubmitOrder, CancelOrder and ChangeOrder with setting Unmanaged = true in the Initialize() is the new unmananged order submission approach in NT7. As there's no signal tracking done by NT, you would need to store IOrder's for further management.

For overfills, there could be an 'inflight' execution case present, i.e the strategy attempted to cancel an order while it was filled.

[ScoobyStoo]

Thanks Bertrand

Hi ScoobyStoo, this is correct using SubmitOrder, CancelOrder and ChangeOrder with setting Unmanaged = true in the Initialize() is the new unmananged order submission approach in NT7. As there's no signal tracking done by NT, you would need to store IOrder's for further management.

That's what I assumed. Store the IOrder returned by the SubmitOrder() method and use that for modifications and cancellations.

For overfills, there could be an 'inflight' execution case present, i.e the strategy attempted to cancel an order while it was filled.

Oh right. So it relates to the rejection of a modification or cancellation request due to simultaneous fill. Got it. That's no problem. I handle all that within my strategy so I'll turn it off.

One further question. When I pass the OrderAction parameter to the SubmitOrder() method do I have to use the SellShort and BuyToCover for short positions? Doesn't really make sense when the order is not being internally managed by NT. Can I just use Buy and Sell regardless of whether the position is long or short?

Thanks

[NinjaTrader_Bertrand]

ScoobyStoo, correct it should not matter, as this method just submits a buy or sell order - so there's no reverse for example as one would be used to with the regular Enter() methods in NinjaScript.

[ScoobyStoo]

Thanks again Bertrand. I am always impressed by the speedy response of NT customer service...



It's been quite a long time since I looked at the NT API but another thing occurs to me...

Given that you don't have states for individual order modifications or cancellations how do you recommend tracking when one gets rejected?

For instance, if I submit a modification that is rejected (e.g. because perhaps the new quantity is rejected by my broker because it exceeds the risk management tolerances of my account) how will I be able to detect it? Will the order state will follow the transitions below?

Modification submitted
OrderState.Working --> OrderState.PendingChange

Modification rejected
OrderState.PendingChange--> OrderState.Rejected

This doesn't really reflect that state of the order because the order has not been rejected, only the modification. The order is still working.

Thanks

[NinjaTrader_Dierk]

A rejection of an order change would reset the order to the last "working" state and would NOT set the order to rejected.

[ScoobyStoo]

OK. I want to be absolutely sure about all this so...

(1) Can you confirm the meaning of various order state transistions:

OrderState.Accepted --> OrderState.Working
The order is begun working at its destination.

OrderState.PendingChange --> OrderState.Working
If the order properties (e.g. quantity, limit price, stop price etc) match the values submitted in the modification request then the modification has been successfully completed.

If the order properties (e.g. quantity, limit price, stop price etc) DO NOT match the values submitted in the modification request then the modification has been rejected.

OrderState.PendingCancel --> OrderState.Cancelled
The cancellation has been successfully completed.

OrderState.PendingCancel --> OrderState.Filled
The cancellation has been rejected as the order had already been completely filled before the cancellation request was processed by the destination. (I can't think of any other reason that a cancellation request would be rejected).

Are these correct?

(2) Can you confirm that OrderState.Rejected should only ever be encountered when an order cannot be placed at its destination (e.g. rejected by broker or exchange) and should never be encountered after the order has been placed.

(3) If I need to know the previous state of an order to be able to infer the meaning of a call to OrderUpdate() from where do you recommend I retrieve the order's previous state. It doesn't seem to be available through the NT API so do I need to persist it myself?

Thanks

[NinjaTrader_Dierk]

>> OrderState.PendingCancel --> OrderState.Filled
I never have seen that with a broker so I would not know

>> OrderState.PendingChange --> OrderState.Working
IOrder has a .Error property which you should check. Comparing "before" and "after" on the Limit/StopPrice and Qty is not the right way.

[ScoobyStoo]

>> OrderState.PendingCancel --> OrderState.Filled
I never have seen that with a broker so I would not know

So what behaviour can I expect to see when a cancellation cannot be processed because an order has been completely filled?

>> OrderState.PendingChange --> OrderState.Working
IOrder has a .Error property which you should check. Comparing "before" and "after" on the Limit/StopPrice and Qty is not the right way.

So will IOrder.Error == ErrorCode.UnableToChangeOrder if the modification is rejected?


What I really need is NT's recommended way of detecting when a cancellation has been rejected and when a modification has been rejected. The rest I can do myself.

[NinjaTrader_Dierk]

There is an error code UnableToChangeOrder for failed modifications and an error code UnableToCancelOrder for failed cancellations.

You would see "Filled" as the broker reports "Filled".

[ScoobyStoo]

There is an error code UnableToChangeOrder for failed modifications and an error code UnableToCancelOrder for failed cancellations.

You would see "Filled" as the broker reports "Filled".

OK, I understand that. You guys are just passing on the messages via the API as you get them from the broker and in the same sequence you get them from the broker.

However, my question hasn't really been answered. What behavior will I see if an order is completely filled before a cancellation request is processed? If an order cancellation request is rejected then does the order remain with a state of PendingCancel? I would imagine not. I would imagine that the destination will send a message to NT indicating that the order has been fully filled, and then send a further message indicating that the cancellation was rejected. Therefore I would expect to see the OrderState change from PendingCancel --> Filled and then the error code changed to UnableToCancelOrder. Is this the case?

As per the docs for NT6.5 I'm assuming OnOrderUpdate() is called whenever any of the properties of IOrder are updated. This includes average fill price, order state etc. This means that in order to establish what has changed I will need to keep a copy of the old IOrder properties for comparison. If this is not the case then please let me know.

I'm sure you appreciate that the sequencing of these things is extremely important to any strategy operating at high frequency.

[NinjaTrader_Dierk]

If an order is Filled then it would not change it's state as it's "terminal" = there is NO way back from "Filled"

[ScoobyStoo]

If an order is Filled then it would not change it's state as it's "terminal" = there is NO way back from "Filled"

I understand that. You aren't understanding my question. If you are reporting a cancellation request rejection through the error code then the order state would not need to change. It will remain filled. My question is how do you report this situation? I can't just leave a cancellation request hanging in the air. If the cancellation rejection is inferred by the change of state to filled then I need to know that so I can handle it then. However, if a further callback to OnOrderUpdate() is made with the error code set then I need to know that so I can handle it differently.

I can't afford to find this stuff out by trial and error. It's critical that I know the sequencing of events that NT raises when it processes incoming order reports.

[NinjaTrader_Dierk]

I fail to understand where you see a problem: the order ultimately is filled which is reported by NT as .Filled no matter what error code you have got BEFORE (!). This means the order is filled, anything else is irrelevant. There is nothing "hanging in the air". Where exactly is the problem?

Also: NT does not dictate the sequence of events, but the broker does. You better have your code prepared for that. NT only guarantees that is there is "no way back" from .Filled.

[ScoobyStoo]

Yes. As I said, I understand that the messages can arrive out of order from the broker. I have coded to several broker/FCM APIs and FIX so I'm familiar with how the messaging works.

There is a layer of abstraction though that NT applies to processing these messages and that is what I am trying to understand.

I need to feed individual events into my strategy relating to the state of every request made with respect to an order (generation, modification and cancellation). Because NT reports on all these events through the OnOrderUpdate() method I am trying to establish a robust way of determining what event has occured causing OnOrderUpdate() to be invoked. To do this I need to evaluate what has changed by holding an old copy of the IOrder object for comparison purposes.

If NT invokes the OnOrderUpdate() method in response to EVERY message received regarding that order then it's easy. It is doesn't then I need to know which messages get suppressed.

Let's make this simple with an example...

If I submit an order cancellation request the OnOrderUpdate() method is invoked and the OrderState is changed to PendingCancel. Yes?

Then if that order is completely filled at the exchange (before the cancellation arrives) the exchange generates an order completion report. NT receives and processes that report and invokes the OnOrderUpdate() method and the OrderState is changed to Filled. Yes?

Then the exchange will receive the cancellation request and generate an order cancellation rejected report. NT receives and processes that report.

My question is:

(1) Does NT invoke the OnOrderUpdate() method again and change the ErrorCode to UnableToCancelOrder?

(2) Does NT do nothing because the OrderState has already been changed to Filled and the order is considered to be in its final state?

I am hoping the answer is 1 because it is easy to process that event. If the answer is 2 then I need to infer the rejection of the cancellation request from the change of order state from PendingCancel to Filled.


Hope this makes sense now.