Apparent bug in intrabar price sequencing

[anachronist]

There seems to be a problem in how NinjaTrader models intrabar movements when finer-resolution data isn't available. I suspect it does no modeling at all.

The problem is this: If an entry order is placed along with automatic target and stop, and one of the exit prices exists on the same bar as the entry price, then both will get filled regardless of the obvious intrabar price movement.

Here's an example showing the problem, during backtest:



Here's the same example, showing correct behavior (strategy running during market replay):



The logic to simulate intrabar movement during backtesting seems broken. It should be as follows:

• Assume the intra-bar movement is open-low-high-close, if the open is in the lower half of the bar, or if the open is in the center and the close is in the upper half.
• Assume the intra-bar movement is open-high-low-close if the open is in the upper half of the bar, or if the open is in the center and the close is in the lower half.

Although I love NinjaTrader and have no wish to go back to TradeStation, I feel compelled to point out that TradeStation figured this out a decade ago, documented in the the TS2000i EasyLanguage manual published in 2001.

I know I can simulate the correct behavior by adding a shorter time frame to my strategy and executing orders on that. However, this isn't feasible with 1 minute bars on a deep history where providers don't maintain tick data. In that case it would be good to have a decent intrabar model to rely on.

Is there an improvment in the works, or is there a way I can improve it myself? I did have a look at DefaultFillType.cs and LiberalFillType.cs, but they seem to be related more on how to handle fills, rather than the sequence in which the fills should occur. I note that those files reference properties 'NextHigh' and 'NextLow', which might be related to this issue. If so, then the logic to determine their values doesn't seem to be working well.

I hope I have documented this problem adequately for your developer team. Let me know if you need further information.

-Alex

[NinjaTrader_Brett]

Hello,

Thanks for the detailed report.

I can confirm that NinjaTrader currently does not do any price movement assumptions in our backtests as assumptions are in fact just assumptions and in no way necessarily reflect reality.

We preferred this method as we just fill the stop instead of the target which understates account performance which gives the user more conservative results which we choose to lean towards the conservative results.,

I have also forwarded your post to the development team and they have assigned it ID 1530.

Thanks you very much for the detailed post.

-Brett

[anachronist]

Brett, thanks, but I think you're misunderstanding the situation. NT's current method doesn't reflect reality either, in fact I argue that it's worse than if NT made intrabar assumptions.

We preferred this method as we just fill the stop instead of the target which understates account performance which gives the user more conservative results.

No, that is incorrect.

Please look at my initial post again. NT's backtest didn't fill the stop. NT filled my target, which gives more optimistic results, not more conservative results.

Look at the price action in the pictures above. You can clearly see that NT's current backtest method is completely wrong and over-optimistic.

The assumptions I described about intrabar price movement would be correct most of the time. Those times when the assumptions aren't correct would be a minority, and they would tend to cancel out.

NTs backtest fills, without attempting to model intrabar price movements, give unrealistically optimistic results, not more conservative results. Why? Two reasons.

1. Stops are larger than targets. Closer targets are statistically more likely than farther-away-stops to appear on the same bar as entry bar.
2. My strategy attempts to go long near the bottom of a bar and short near the top end. This gives the target price even more room to allow it to appear on the entry bar. The stop is unlikely to be on the entry bar.

If the target price occurs on the entry bar more often than the stop price, backtesting will usually show unrealistically optimistic, non-conservative results. I daresay that most daytrading strategies that use stops and targets will have bigger initial stops than targets.

-Alex

[NinjaTrader_Brett]

Thanks for the feedback.

Will forward it.

-Brett