Nothing makes sense

[PabloDea]

After several years of developing strategies, primarily focusing on strategies with small tick profits, I find myself facing the problem that my ideal entry prices in my strategies differ significantly from the actual entry prices in the real market. This directly renders my backtesting useless.

Every variable in the backtesting results determines whether a strategy is successful or not, and that price difference between backtesting and real-life entry prices changes the whole game. This leads me to question the usefulness of the backtesting system —it either serves no purpose or only applies to strategies with large profits that can tolerate the gap between the ideal entry price and the real price. Even then, the backtesting data wouldn't be accurate.

For example, I enter a long position on the NQ at 14333, and it actually enters at 14335, with a 5-point difference. Within that difference, my entire strategy is invalidated, meaning it enters outside of any parameter considered in my strategy. This happens much more frequently than I anticipated. Every day.

I hope I don't sound aggressive, but I'm tired of struggling with this issue and the existential problem of whether all of this makes sense.

Just to clarify, I have tried multiframes with 1 tick series, set the order fill resolution to high, etc. The problem lies elsewhere.

Is there anything that can be done about it, or do I simply resign myself to the fact that the system is designed for larger profits, but the values generated by the backtesting would still be incorrect?

[NinjaTrader_ChrisL]

Hello, thanks for writing in. The only way to improve order fill performance and accuracy is to add the 1 tick series to your script and submit your orders to this series. The 1 tick series is the highest resolution possible for a backtest. We have a guide on implementing this and also an example on the help guide page.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/strategy-development/94098-isfirsttickofbar-vs-onbarclose-for-backtest-live

https://ninjatrader.com/support/helpGuides/nt8/NT%20HelpGuide%20English.html?backtesting_ninjascript_strate.htm

[QuantKey_Bruce]

From a distant perspective, my comment would be that you may be trying to get in with such small scalping trades that small differences ruin the performance. You can try to enter with limits instead of market etc. and possibly miss some trades but trying to control slippage on the way in, and that may help (though sometimes, this gets you into only the trades that are going the wrong way). What some do in response to the conundrum you describe is aim higher - aim for longer time frames and much larger trades where there is less noise and a tick or two here or there makes no difference. For instance, if you have a 5-tick difference on an entry on NQ daily bars or even 60-minute bars, it is unlikely to matter that much if your system is even marginally profitable. But if you're scalping tiny Renko or tick bars, that can be a deal killer. Each system has different slippage characteristics that have to be measured during deployment (or evaluation for deployment) and it is not uncommon for systems to work in backtesting and fail live because, for instance, they're getting in with zero seconds lag in back-testing while in live trading it takes 1/2 second or whatever and that kills it. That happens all the time - one way to avoid that happening to you is to aim for longer-term trades than this so 1/2 second is inconsequential. Or, you can do everything possible to reduce that 1/2 second and mitigate those factors but bear in mind you're competing with people with seats on the exchange and colocation such that their latencies are in the microseconds and their round-trip costs are in the pennies. There are lots of lucrative trading opportunities, but at the high-frequency end it is tough to compete with the big boys so while those kinds of strategies are not impossible they are a lot more painful in almost every way.

[PabloDea]

ChrisL, that's the first thing I tried, without good results. In strategies with small tick profits, even the slightest change can turn the profit factor from positive to negative, or worse, from negative to positive. It's very frustrating.

Bruce, thank you for your comment, it's valuable to me.

I have come to the conclusion that this type of trading, the one offered by NinjaTrader or most systems for end users, only works well for larger timeframes or, more accurately, for longer profits where the deviations between what you want and what actually happens don't affect your strategy as much.

Unfortunately, it took me a lot of time and money to reach this conclusion.