Multi Instrument Backtest Problem

[SoulSurfer]

Please attach the script that you ran this test on so I can further investigate this

As I explained it's only slightly changed from my original version. However the latest version is attached.

To restate my earlier question once more, is there any way for my CustomFillType to determine which instrument triggered the current invocation, since evidently for any particular order, and whilst it remains unfilled, Fill(Order order) is invoked for every instrument in the basket, not just order.Instrument?.

[NinjaTrader_CalH]

SoulSurfer,

I dug a little deeper here and noticed the mistake I did in my test. I used a BarsInProgress filter for all the orders and didn't have an order that was left working for the next bar update.

I adjusted my previous post to match what is the Core design.

Each bar update will try to identify a working order. If there is a working order it will send a request to the FillType. Note that each OBU print is at the end of the OnBarUpdate() method.

I do apologize for the confusion. You can see which BarsInProgress requested the FillType by running a Print check at the end of OBU and see when the FillType is called.

[SoulSurfer]

OK - It seems like we're on the same page at last

The simplest way that I can see to solve my problem is if my custom Fill() can be made aware of "which BarsInProgress requested the FillType". Is that information already available inside Fill()? If not, is it possible (and preferably simple) for a modified OBU to make that information available to Fill()?

[NinjaTrader_CalH]

There is no way to determine in the FillType which OBU was called for the fill.

The only suggestion I have is that you use the print method to see where in your test the FillType was called and when a order was filled.

[SoulSurfer]

There is no way to determine in the FillType which OBU was called for the fill

Adding an extra diagnostic Print() statement does nothing to help me extract even vaguely accurate backtest results from the NinjaTrader strategy analyzer though.

Would it be accurate to say that NinjaTrader is currently useless for my particular use case?

Assuming the answer to that question is yes, can you explain the rationale behind the current way in which "the NT Core logic is designed"? Is there any chance that it might be redesigned, at the very least to add some means whereby the FillType can determine which OBU caused it to be invoked?

[NinjaTrader_JC]

Hello SoulSurfer,

Could you clarify why exactly it is important to see which OBU event invoked the Fill? This much handling does not seem very dynamic and almost form fitting for backtest results by the sounds of it.

My guess is that it has to do with your symptom #3

3. Backtest results vary depending upon the order in which the instruments are assigned to the various BarsInProgress.

If you could clarify the varied results there maybe a way to resolve this without the need for a custom fill type.

Happy to be of further assistance.

[SoulSurfer]

Hi JC,

Could you clarify why exactly it is important to see which OBU event invoked the Fill?

Hopefully Cal understands the problem by now, but to reiterate, if I knew which BarsInProgress invoked the Fill() I could easily craft a CustomFillType to ignore all potential fills until (by way of example) OBU 2 had been run, which would have updated my exits for BarsInProgress=2, which change every day.

Does that make sense now?

I can see a potential way around the problem, but it would involve 25 years of one minute data for a large number of instruments. Firstly that's a bit hard to come by, and also eats up a lot of CPU cycles.

[NinjaTrader_JC]

Hello SoulSurfer,

That is what I got out of your first statement but I see no reason why this would be helpful. In a Multi-Instrument scenario waiting for another Bar Update especially on another instrument has no point since it is looking at different data. $GBPUSD is going to fill on the $GBPUSD price. $GBPCAD is going to fill on the $GBPCAD price.

From talking to our Development team it looks as if they are looking at changing the way orders are filled so the next major release of NinjaTrader will be different than NinjaTrader 7 but I don't have any further information than that.

[SoulSurfer]

Hi JC,

That is what I got out of your first statement but I see no reason why this would be helpful. In a Multi-Instrument scenario waiting for another Bar Update especially on another instrument has no point since it is looking at different data. $GBPUSD is going to fill on the $GBPUSD price. $GBPCAD is going to fill on the $GBPCAD price.

By way of example, assume one moves one's stop at the end of each day to follow some assumed "trend line". This operation is performed in OnBarUpdate() each and every day for each and every instrument. Assume further that $GBPUSD corresponds to BarsInProgress=0 and $GBPCAD corresponds to BiP=1. If Mr. Market doesn't cooperate (which has been known to happen from time to time) then $GBPUSD will be stopped out at today's level, whereas $GBPCAD will be stopped out at yesterday's level, since the first fill check for BiP 1 occurs after OBU 0. Reverse the order of the instruments leaving everything else unchanged and the NT strategy analyzer reports different trades and a different equity curve. I don't know about you, but personally I find that extremely unhelpful. Strange to relate, but so does our client!

From talking to our Development team it looks as if they are looking at changing the way orders are filled so the next major release of NinjaTrader will be different than NinjaTrader 7 but I don't have any further information than that.

I look forward to discovering what changes NinjaTrader 8 ultimately brings in this area. In the meantime can you suggest a workaround for this "feature" in NinjaTrader 7?

[NinjaTrader_JC]

Hello SoulSurfer,

Thanks for the example. So I think we are going about this the wrong way. NinjaTrader 7's event driven system is always going to be calling the BIP 0 first so I think creating a custom fill type is mostly going to be a waste of time since there is not a good way to be able to pass any information to the custom fill type.

The best way to get your results that you are looking for would be to either add an intraday series so that it will get called before the Daily series that you are looking at or if you use daily data to go back farther then you may want to only submit orders to one series at a time if you use data for both series at once. Essentially the second way would be to separate out the logic so that your Data that you use for your calculations comes in first and then you can submit orders to the second series to ensure that your data comes in first. This would require running two different strategies or changing the code after running the strategy to compare the results but may be the best and easiest way to get the results you are looking for.

Let me know if that makes sense.

[SoulSurfer]

Hi JC,

Let me know if that makes sense.

It makes sense in so far as I already suggested that:

I can see a potential way around the problem, but it would involve 25 years of one minute data for a large number of instruments. Firstly that's a bit hard to come by, and also eats up a lot of CPU cycles.

I also much prefer to use exactly the same code in production as in testing, so adding extra data series that are redundant in live trading is far from an ideal solution from my perspective. However if that's the only realistic workaround I'll give it a try.

[SoulSurfer]

JC,

Following some experimentation with code completion and the debugger I think I've come up with an elegant solution to my problem. There seem to be some undocumented (and presumably unsupported?) and otherwise unused variables of the form Strategy.VariableN that allow me to pass the BiP for the most recent OBU into my CustomFillType(). Then I can ignore any potential fills until I'm certain that the relevant exit orders have been updated.

Can you see any reason why that wouldn't work, as a temporary workaround at least?

[NinjaTrader_JC]

Hello SoulSurfer,

Variables0-9 is supported. They are just predefined variables mainly used for the Strategy Wizard so that users can set those variables to values to use later on.

CustomFillTypes are not supported in NinjaTrader 7, but with that said it may do what you are looking for.