MTF in an Indicator

[Harry]

Have tested yesterday but could not consistently reproduce the problem of double runs of OnBarUpdate(). I still continue to explore the issue.

[NinjaTrader_JoshP]

Harry,

Thank you for your efforts. It is appreciated. I have been testing it on my end as well to no avail just yet either.

[Harry]

Harry,

Thank you for your efforts. It is appreciated. I have been testing it on my end as well to no avail just yet either.

But I have definitely had that problem as documented below, so I am sure it will come back, as the FOREX price spikes and volume spikes also come back regularly.

I think it might be related to the amount of real-time data temporarily stored.

[NinjaTrader_JoshP]

Understood, but unfortunately there is little I can do without being able to reproduce. We will keep our eyes on it for more information that could help us isolate down the scenario, but until then I have no further information I can provide.

[Harry]

Hi Josh,

I have now been able to reproduce this, I think it depends on the indicator and the number of incoming ticks. I used your code snippet in the first line of OnBarUpdate().

First finding: There are really problems with the first real-time bar, the real-time ticks are blended into the historical ticks in a very strange fashion. This may also explain the spike problem :

True 24.03.2011 20:15:52 0 True
False 24.03.2011 20:15:55 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 True
True 24.03.2011 20:13:53 0 True
True 24.03.2011 20:14:00 1 True
True 24.03.2011 20:14:14 0 True
True 24.03.2011 20:14:30 0 True
True 24.03.2011 20:14:53 0 True
True 24.03.2011 20:15:00 1 True
True 24.03.2011 20:15:02 0 True
True 24.03.2011 20:15:12 0 True
True 24.03.2011 20:15:26 0 True
True 24.03.2011 20:15:33 0 True
True 24.03.2011 20:15:42 0 True
True 24.03.2011 20:15:46 0 True
True 24.03.2011 20:15:52 0 True
False 24.03.2011 20:15:55 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 True
True 24.03.2011 20:13:53 0 True
True 24.03.2011 20:14:00 1 True
True 24.03.2011 20:14:14 0 True
True 24.03.2011 20:14:30 0 True
True 24.03.2011 20:14:53 0 True
True 24.03.2011 20:15:00 1 True
True 24.03.2011 20:15:02 0 True
True 24.03.2011 20:15:12 0 True
True 24.03.2011 20:15:26 0 True
True 24.03.2011 20:15:33 0 True
True 24.03.2011 20:15:42 0 True
True 24.03.2011 20:15:46 0 True
True 24.03.2011 20:15:52 0 True
False 24.03.2011 20:15:55 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 True
False 24.03.2011 20:15:55 0 False
False 24.03.2011 20:16:00 1 False

Second Finding: NinjaTrader regularly violates the alternance of calling the first and the second BarSeries on incoming ticks. In the case I found, the double BIPs always came be groups of three about every 10 seconds:

False 24.03.2011 20:16:00 1 False
False 24.03.2011 20:15:56 0 False
False 24.03.2011 20:15:56 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 False
False 24.03.2011 20:15:56 0 False
False 24.03.2011 20:15:56 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 False
False 24.03.2011 20:15:56 0 False
False 24.03.2011 20:15:56 0 True DOUBLE BIP
False 24.03.2011 20:16:00 1 False
False 24.03.2011 20:15:56 0 False

False 24.03.2011 20:17:00 1 False
False 24.03.2011 20:16:03 0 False
False 24.03.2011 20:16:04 0 True DOUBLE BIP
False 24.03.2011 20:17:00 1 False
False 24.03.2011 20:16:03 0 False
False 24.03.2011 20:16:04 0 True DOUBLE BIP
False 24.03.2011 20:17:00 1 False
False 24.03.2011 20:16:03 0 False
False 24.03.2011 20:16:04 0 True DOUBLE BIP
False 24.03.2011 20:17:00 1 False
False 24.03.2011 20:16:05 0 False
False 24.03.2011 20:17:00 1 False

The test was conducted on a 10-tick chart (condensed ticks) for ES, with the second called bar series being a 1 minute series.

[NinjaTrader_JoshP]

Harry,

How do you achieve it to go from Historical to real-time back to historical in finding #1? That would likely be the critical piece to this puzzle.

After loading 20 times in a row I have never once ran into such a scenario and would need to reproduce that to continue.

[Harry]

Hi Josh,

I have just added this to my indicator. The indicator is work in progress, not terminated. I can send you a copy, if you treat it confidentially. I think you will be able to reproduce the problem, if you connect to Interactive Brokers. Problem may depend on tick speed.

The problem arises, when I press F5. Real-time data then becomes historical data and you have

-> historical data in the historical data base
-> historical data (collected real-time data), which is not in the historical data base but somewhere in a cache (I do not save real-time data as historical)
-> real real-time data

You can also connect to my machine and watch during RTH.

[Harry]

Another test with EURUSD during the after hours:

True 24.03.2011 22:57:00 1 True
True 24.03.2011 22:57:08 0 True
True 24.03.2011 22:57:13 0 True
False 24.03.2011 22:57:13 0 True DOUBLE BIP
False 24.03.2011 22:58:00 1 True
True 24.03.2011 22:54:20 0 True
True 24.03.2011 22:54:34 0 True
True 24.03.2011 22:54:42 0 True
True 24.03.2011 22:54:58 0 True
True 24.03.2011 22:55:00 1 True
True 24.03.2011 22:55:09 0 True
True 24.03.2011 22:55:25 0 True
True 24.03.2011 22:56:00 1 True
True 24.03.2011 22:56:31 0 True
True 24.03.2011 22:57:00 1 True
True 24.03.2011 22:57:08 0 True
True 24.03.2011 22:57:13 0 True
False 24.03.2011 22:57:13 0 True DOUBLE BIP
False 24.03.2011 22:58:00 1 True
True 24.03.2011 22:54:20 0 True
True 24.03.2011 22:54:34 0 True
True 24.03.2011 22:54:42 0 True
True 24.03.2011 22:54:58 0 True
True 24.03.2011 22:55:00 1 True
True 24.03.2011 22:55:09 0 True
True 24.03.2011 22:55:25 0 True
True 24.03.2011 22:56:00 1 True
True 24.03.2011 22:56:31 0 True
True 24.03.2011 22:57:00 1 True
True 24.03.2011 22:57:08 0 True
True 24.03.2011 22:57:13 0 True
False 24.03.2011 22:57:13 0 True DOUBLE BIP
False 24.03.2011 22:58:00 1 True
False 24.03.2011 22:57:16 0 False
False 24.03.2011 22:58:00 1 False
False 24.03.2011 22:57:16 0 False
False 24.03.2011 22:58:00 1 False

So it is easily reproducible.

[NinjaTrader_JoshP]

Harry,

Please send in your file and we will use it in confidence. Please send to support [at] ninjatrader [dot] com with Attn: Josh in the subject line and mention this thread. Thank you.

[Harry]

Harry,

Please send in your file and we will use it in confidence. Please send to support [at] ninjatrader [dot] com with Attn: Josh in the subject line and mention this thread. Thank you.

Thank you for your answer, Josh. I am currently further testing into this.

The first finding of post # 35 below is actually nothing special. I had three instances of the same indicator running on my chart, so they switched from historical to real-time one by one at different times. This is not unexpected.

This also explains, why when a Double BIP (BarsInProgress) occurred, there were always three consecutive Double BIPs. This was because the problem occured on all three instances of the indicator consecutively. I have now modified your code to show, which of three instances is called by OnBarUpdate(), and I am further testing.

I had some Double BIP bars again, but not sure what are the conditions required to cause them. I will come back once I have finished testing on my side.

[NinjaTrader_JoshP]

Harry,

Thank you for your tenacity in this matter.