NT8 reading of strategy position Incorrect/discrepancy

[moewqasem]

Hello,
Not sure if this should go here or in strat developement. I identified this problem when my live strategy was missing certain entries.


Default Settings:
Calculate = Calculate.OnBarClose;
EntriesPerDirection = 1;
EntryHandling = EntryHandling.UniqueEntries;
Primary data series ES 60mins, no additional data series

I found out the entry is missed in live trading, if in the scenarios where both these conditions happen:
- 1 unique position is open, exited on any bar.
- Entry signal on the following bar close bar

In back test this position is taken as expected
In Market replay this position is taken as expected
In LIVE TRADING this position is NOT taken, it is missed

The reason as i suspect is because the strategy thinks the position still exists and there and must not exceed the 1 position per unique entry.

To diagnose: I added a secondary data series which is also ES 60 mins. and the below code for prints under onBarUpdate():

if (CurrentBars[0] < 1 || CurrentBars[1] < 1 )
return;

if (BarsInProgress == 1) Print("BarsInProgress[" +BarsInProgress+ "]: "+Times[1][0] + ": "+ Position.MarketPosition+": "+Position.Quantity);



if (BarsInProgress != 0)
return;


Print("BarsInProgress[" + BarsInProgress+ "]: "+Times[0][0] + ": "+ Position.MarketPosition+": "+Position.Quantity);

Here are the resulting Prints at one of the problem areas in back test and in Replay:


Backtest run: 60min and 60min


Replay run: 60min and 60min



Backtest results: identical positions on identical timestamps (as expected with no missing trades).
Replay summary(see below)


Observations:
in Replay , for the same time stamps we see a different position if we print from Primary vs if we print from Secondary.. i know secondary bar update is processed following the processing of the primary bar update and might be the cause of this difference.

The point here though is that the Secondary series prints are the updated/ correct/ most recent ones and the Primary prints are delayed information. the hypothesis is that this is what causes the trade to be missed in live trading as the missed trade in Live trading is always on the bar immediately following an exit.
to prove hypothesis i changed the code to run on BarsInProgress[1]

Summary of Results:
back test and market replay submitting bars to BarsInProgress[0] or BarsInProgress[1] result in the same trades which are correct despite the print having different values!
Only in live trading these trades are missed, what gives?
delay for position fill update in live trading to come back from my broker (IBKR)?
How do i fix this? what do i need to change in my code since i can not replicate the issue to fix it in the back test or in replay even though i replicated the issue in the prints?
Thanks for reading and excuse my long post

[NinjaTrader_ChelseaB]

Hello moewqasem,

There are differences between backtest and real-time.

1-tick intra-bar granularity (and TickReplay if using Calculate.OnEachTick/.OnPriceChange) can help minimize those differences.

https://support.ninjatrader.com/s/article/Developer-Guide-Improving-backtest-order-fill-accuracy-with-intrabar-granularity

To understand what specifically is causing differences, write to file the data and condition values with labels for each value and comparison operator.
Then compare the output from the real-time and backtest.

Below is a link to a support article which provides direction, a sample script, and a video.

https://support.ninjatrader.com/s/article/Developer-Guide-Comparing-Strategy-Results