Backtest not giving representative results

[samish18]

I am running a strategy on a 1-tick bar and comparing the same time period running live to backtest, there are absolutely no similarities - there are no common trades between live and backtest

[NinjaTrader_Jesse]

Hello samish18,

To find out what is going on you can add debugging prints into the script to see why the conditions to trade are becoming true when they are which is leading to discrepancies. We have a guide that goes over how to compare realtime and backtest results and diagnose differences here:

https://support.ninjatrader.com/s/article/Developer-Guide-Comparing-Strategy-Results?language=en_US

[samish18]

This is the strategy. It uses limit orders so I don't know where I would put prints. There is absolutely no connection between live and backtest

[NinjaTrader_Jesse]

Hello samish18,

The strategy you provided has no conditions for entry, it just enters as data processes. Due to that it would not be expected for this to match a backtest, there are not specific points where a trade should be placed and how long a trade stays open will depend on the market movement and where the targets fill. The trade will be immediately opened again once they do fill.

For this to be similar to a realtime test you would need to have started on the exact same tick in both tests however that may still have differences based on using the historical fill engine vs a live market for the fills. If there was any difference in fill price for the entry or targets that would throw off the test because the strategy has no specific entry conditions.

[samish18]

I trade would be expected when price jumps up to fill the limit order and is open until it hits stop loss or take profit.

[NinjaTrader_Jesse]

Hello samish18,

I am not certain what you are trying to say here, can you clarify what you are asking?

Based on the strategy you provided I would expect it to be very difficult to get similar results to a realtime test for a number of reasons as explained in my last reply. The tests would need to start on the exact same tick to be remotely similar and there still could be differences because of how the historical fill engine works vs how a realtime market fills orders. The main reason for the differences is that there is no specific condition to enter an order. A simple example would be if your backtest started on a tick that is at 12:00pm but your realtime test starts on a tick that is 12:01:01pm that will cause differences in where the first entry fills and where the targets later fill. For every trade past that point the differences will grow larger and larger because the tests were not equal.

[samish18]

Wouldn't any difference between the live and backtest data set be cancelled after any period where no orders were being filled (such as 5-6 pm EST for ES)? Since the conditions for order entry are dependent only on the current bar

[NinjaTrader_Jesse]

Hello samish18,

If you left the strategy running into the next session in theory at that point it would match a backtests assuming the backtest had started at the exact same tick as the realtime chart. Both tests will have an order submitted immediately at the start of the session. Beyond that point you could still end up seeing differences because there are no rules for the entry order. If any fill past that point does not match exactly that will cause it to go out of sync again.

You would need to have specific entry conditions so that in both tests the entry starts at the exact same point every time, that would then allow the targets to fill at similar points and not re enter immediately after. Once the condition becomes true again at a later time both tests would submit a new entry at the exact same time. The main take away is that there is some space between the exit and the entry, your current logic does not have any space between the exit and the entry so if the exit happens differently the next entry will be out of sync.

[samish18]

The entries and exit are usually quite close (only in the market for a couple seconds) and entries execute maybe every couple of hours. Similar to the two syncing up when entering a new session, wouldn't they sync up over an hour of no trades being executed?

[NinjaTrader_Jesse]

Hello samish18,

That depends if the exits fill at the exact same price so a new entry can be submitted on the following update. If the exits do not fill at the same price/time then the next entry will be different between the tests. Once that happens one time the difference will get larger and larger with every new trade because the entries will not be in sync.

You would need to make a more specific condition to enter so that the entries are always placed at the same point. Even if the targets don't fill identically that still allows the next set of entries to be placed based on something other than bar processing.

Here is a very simple example of why the strategy in its current state would be very difficult to have it sync up

Both strategies place an entry on bar 0

Realtime Strategy has a target fill on bar 1 at a price of 100.00
Realtime Strategy submits a new entry on bar 2

Historical Strategy has a target fill on bar 2 at a price of 100.01 (we will assume there was a difference in how the historical fill engine filled the limit order so it was filled 1 bar later)
Realtime Strategy places a new entry on bar 3

There was a separation of where the next entries are so now going forward this separation is going to get larger and larger as the market requirements for the targets to be filled are different between each trade.

[samish18]

And then if the entry submitted by the realtime strategy is not filled, it submits another order on bar 3 identical to the historical strategy

[NinjaTrader_Jesse]

Hello samish18,

That's a possibility but that's besides the point, the strategy as it stands has a very high probability of encountering differences when comparing to a backtest. All that I can suggest is to make more specific entry conditions so you can ensure the entries are happening identically. Without doing that there would not be a solution to prevent differences from accumulating over time to create larger differences.