Why is Historical flag true on launch with live enabled?

[molecool]

I've got a routine that does profit verification and it runs on FirstTickOfBar. When Historical is true then it's adding 1 to my index as I'm doing profit verification on a tick series but am checking the index of my minute 0 series.

Anyway, my problem is that I get Historical = true on my transition candle when I turn my strategy on. And yes, I have immediately submit 'live working historical orders' set in my NinjaScript tab. My strategy has min bars required set to 0.

Nevertheless I get historical as true when I would need it immediately to be false on the current live candle, i.e. when it's the very last tick on the chart.

Is there any way for me to fix this? I've tried pretty much everything and I'm about to give up

[NinjaTrader_Bertrand]

molecool, so you run this script then with CalculateOnBarClose = false?

[molecool]

molecool, so you run this script then with CalculateOnBarClose = false?

Yes of COURSE ;-)

Based on your response may I assume that this should not happen? Well, if so - it is - on three different NinjaTrader deployments. So I do not think this is based on some hick up on my end.

Any input is welcome as I cannot find a solution for this - it's been months now I'm trying to address this. Many thanks.

[NinjaTrader_Bertrand]

Please give this simple single series script a test run - how you would startup the strategy (WaitUntilFlat or not) should not matter. I would expect the transition candle to show Historical false if the OnBarUpdate is processed for live seen ticks. That's the output I'm getting here on my end, testing for example ES 09-13 15 min on ETH sessions. Could it you remove this effect by processing only in FirstTickOfBar?

[molecool]

Some progress...

Please give this simple single series script a test run - how you would startup the strategy (WaitUntilFlat or not) should not matter. I would expect the transition candle to show Historical false if the OnBarUpdate is processed for live seen ticks. That's the output I'm getting here on my end, testing for example ES 09-13 15 min on ETH sessions. Could it you remove this effect by processing only in FirstTickOfBar?

Without running the example I can tell you right now that it is the FirstTickOfBar flag that is causing this. I'm saying this as I have run into this problem on a previous occasion in the same strategy. That time I was able to find a work around. This time around I am having some success with using the current replacement flag:

bool[] lastBars = new bool[BarsArray.Length];
     for (int x = 0 ; x < BarsArray.Length; x++) {
	  lastBars[x] = CalculateOnBarClose ? CurrentBar == Count - 2 : CurrentBar == Count - 1;
}
			
bool isHistoricalBOT = lastBars[1] ? Times[0][0] <= Times[1][0] : true;

That is placed at the beginning of my OnBarUpdate method. I don't think it's reliable in 100% of the cases but in the majority of the cases I do get a false flag if it is the very last candle on the chart AND if the 0 minute series is ahead of the 1 tick series.

If you have a better suggestion I'm all ears ;-)

[NinjaTrader_Bertrand]

Ok, yes then I would expect the outcome you saw happen for Historical - as processing that in FirstTickOfBar is like working COBC = true (as closing tick of old is the same as opening of new in our event based approach).

[molecool]

Ok, yes then I would expect the outcome you saw happen for Historical - as processing that in FirstTickOfBar is like working COBC = true (as closing tick of old is the same as opening of new in our event based approach).

I don't think I am getting what you're saying. Could you please post a more elaborate explanation? When I turn on a strategy NinjaTrader labels the transition candle's current (ongoing) tick as FirstTickOfBar - why is that?

[NinjaTrader_Bertrand]

molecool, when you enable or reload your script - would the chart then not be scrolled to the very right / CurrentBar candle?

[molecool]

This suggests that your Historical flag is broken

Alright folks - I think I may have conclusive evidence that NinjaTrader's Historical flag is broken. Please run the enclosed strategy on your end.

This is some of the output I am getting:

**** OUTSIDE ****
Zero series time: 7/30/2013 11:15:00
One series time: 7/30/2013 11:18:44
isHistoricalBOT = True
Historical = True
Times[0][0] <= Times[1][0] = True
***************
**** OUTSIDE ****
Zero series time: 7/30/2013 11:15:00
One series time: 7/30/2013 11:18:45
isHistoricalBOT = True
Historical = True
Times[0][0] <= Times[1][0] = True
***************
**** OUTSIDE **** <<<<< Here is when I turned it on....
Zero series time: 7/30/2013 11:20:00
One series time: 7/30/2013 11:18:45
isHistoricalBOT = False
Historical = True
Times[0][0] <= Times[1][0] = False
***************
**** OUTSIDE ****
Zero series time: 7/30/2013 11:20:00
One series time: 7/30/2013 11:18:49
isHistoricalBOT = False
Historical = False
Times[0][0] <= Times[1][0] = False
***************
**** OUTSIDE ****
Zero series time: 7/30/2013 11:20:00
One series time: 7/30/2013 11:18:49
isHistoricalBOT = False
Historical = False
Times[0][0] <= Times[1][0] = False

And I double checked that 'immediately submit live working historical orders' is enabled.

[koganam]

Alright folks - I think I may have conclusive evidence that NinjaTrader's Historical flag is broken. Please run the enclosed strategy on your end.

This is some of the output I am getting:



And I double checked that 'immediately submit live working historical orders' is enabled.

So it looks then like this is saying the Historical is true for the 0 series on the FirstTickOfBar?

[molecool]

So it looks then like this is saying the Historical is true for the 0 series on the FirstTickOf Bar?

This is saying something else. If you run the example you'll see that I moved the code outside the FTOB condition - you can test with it as well if you like. What surprises me is that the Historical flag is lagging my own time based condition. I am only checking that condition on the very last bar of the series. And it apparently triggers before the Historical flag switches to false. Or in other words the Historical flag seems to lag by one tick.

Now regarding the HIstorical inside the FTOB that's a different story. I can understand why it can be still true in there, which is why I need my own. But outside that's a different story.

[NinjaTrader_Bertrand]

Same as you would see from my example posted, Historical shows true for the last seen tick historically loaded for your developing bar, which is expected. On your first truly realtime seen tick > it updates to false. So you can see Historical = true for the currently building transition bar holding a future (end of bar) timestamp.

[molecool]

Same as you would see from my example posted, Historical shows true for the last seen tick historically loaded for your developing bar, which is expected. On your first truly realtime seen tick > it updates to false. So you can see Historical = true for the currently building transition bar holding a future (end of bar) timestamp.

I disagree - if you look at my example you'll realize that my flag switches as soon as the Times[0][0] timestamp is larger than Times[1][0]. Yours lags by one tick. Why would that be expected? BTW this is OUTSIDE the FTOB condition.

[NinjaTrader_Matthew]

mole,

In your example, keep in mind that BIP = 0 will always be updated first. The transition seen on BIP 0 is from 11:15:00 to 11:20:00. The tick series (BIP 1) time is 11:18:45 in both calls.

That would be the historical tick bar time when the BIP 0 updated and is the same as the previous call. The next print is the first tick we get on the real-time series at 11:18:49, not 11:18:45.

What you're comparing is the last historical tick called for the bar that is being built. This would have a historical true state.

I think if you add a BarsInProgress print to your test, you'll see what I mean. For example, please see the attached and you'll see that your isHistoircalBOT remains true on BIP 1, although we're getting real-time updates from the tick series.

[molecool]

mole,

In your example, keep in mind that BIP = 0 will always be updated first. The transition seen on BIP 0 is from 11:15:00 to 11:20:00. The tick series (BIP 1) time is 11:18:45 in both calls.

That would be the historical tick bar time when the BIP 0 updated and is the same as the previous call. The next print is the first tick we get on the real-time series at 11:18:49, not 11:18:45.

What you're comparing is the last historical tick called for the bar that is being built. This would have a historical true state.

I think if you add a BarsInProgress print to your test, you'll see what I mean. For example, please see the attached and you'll see that your isHistoircalBOT remains true on BIP 1, although we're getting real-time updates from the tick series.

Yes, I have noticed that as well. However it is dong what I would expect inside BIP=0. When I turn on my strategy the flag I use (Historical or isHistoricalBOT) determines which index I use to look at the previous candle. In live mode I need it to be 1 and in historical mode I need it to be 0. Now, if I use your flag then my strategy looks at the wrong candle and potentially closes out my trade too early. My flag seems to work inside of BIP=0 and with FTOB. Quite frankly why exactly that is - I don't fully understand.

What you are saying makes sense intellectually but my results show that using your Historical flag will not work in the scenario I describe. I still do not understand what you mean when you say that I am looking at the last historical tick bar being built. Well, that is the first bar my strategy encounters when I turn it on and that's the one that counts for me at that moment. Do I make any sense??