onMarketData() vs OnBarUpdate()

[joanrocagas]

Hi,

first post here. I've started to use NT8 and I'm liking it a lot.

I'm developing my first custom indicator and I'm struggling a bit trying to understand how to use the funcitions onMarketData and onBarUpdate.

The indicator I'm developing uses tick replay. I've started looking into the system indicator BuySellVolume source code.

I see a problem in the code below (from the BuySellVolume indicator)

protected override void OnMarketData(MarketDataEventArgs e)
		{			
			if(e.MarketDataType == MarketDataType.Last)
			{				
				if(e.Price >= e.Ask)
				{
					buys += e.Volume;
				}
				else if (e.Price <= e.Bid)
				{
					sells += e.Volume;
				}
			}	
		}
		
		protected override void OnBarUpdate()
		{
			if (CurrentBar < activeBar || CurrentBar <= BarsRequiredToPlot)
				return;

			// New Bar has been formed
			// - Assign last volume counted to the prior bar
			// - Reset volume count for new bar
			if (CurrentBar != activeBar)
			{
				Sells[1] = sells;
				Buys[1] = buys + sells;
				buys = 0;
				sells = 0;
				activeBar = CurrentBar;
			}

			Sells[0] = sells;
			Buys[0] = buys + sells;
		}

In the function onBarUpdate, in the conditional

...
if (CurrentBar != activeBar)
			{
				Sells[1] = sells;
				Buys[1] = buys + sells;
				buys = 0;
				sells = 0;
				activeBar = CurrentBar;
			}
...

Why is sells and buys assigned to the previous bar? Could not be possible that if onMarketData was already executed for the first tick of the new bar, the sells and buys variables would already contain the volumes of the previous bar plus the volume of the first tick of the current bar?

How can we know that onMarketData has added the volumes up to the last tick of the previous bar?

[NinjaTrader_AlanP]

Hello joanrocagas,

Thank you for your note.

OnMarketData will always be called after OnBarUpdate. From the helpguide:

7.The OnMarketData() method is expected to be called after OnBarUpdate()

https://ninjatrader.com/support/help...marketdata.htm

While OnMarketUpdate is called first, OnBarUpdate is where the new bar is created and thus the code within OnBarUpdate is ran before OnMarketData is ran again.

The scenario you inquire about would not be possible.

Please let us know if you need further assistance.

[joanrocagas]

Hello joanrocagas,



OnMarketData will always be called before OnBarUpdate. From the helpguide “The OnMarketData() method is guaranteed to always be called before OnBarUpdate()”

https://ninjatrader.com/support/helpGuides/nt8/en-us/?onmarketdata.htm

.

This is a difference between NT8 and NT7, isn't it?

In NT7, in the docs says:

With NinjaTrader being multi-threaded, you should not rely on any particular sequence of events like OnMarketData() always being called before OnBarUpdate() or vice versa

So, does NT8 syncronize onMarketData and onBarUpdate threads?

[NinjaTrader_AlanP]

Hello Joanrocagas,

Yes, the instrument thread triggers OnBarUpdate and OnMarketData, any data driven events are triggered from the instrument thread.

Please let us know if you need further assistance.

[joanrocagas]

onBarUpdate executes BEFORE onMarketData when Calculate = onEachTick

which in my opinion is just wrong, it will make your indicator give wrong results
please see

http://ninjatrader.com/support/forum/showthread.php?t=94673

[cls71]

I have also been able to verify that the OnBarUpdate method is executed before that OnMarketData method. But this has been happening since the first beta. This behavior is inverse to that of NT7.

I hope you do not change the behavior now because at this point the impact would be high in many NT8 third party developments based on OrderFlow.

joanrocagas, you can detect the first tick of the bar in OnBarUpdate, update a boolean and use it into OnMarketData when the Last object is received.

[joanrocagas]

joanrocagas, you can detect the first tick of the bar in OnBarUpdate, update a boolean and use it into OnMarketData when the Last object is received.

And what is the purpose of that?, I will not be able to update the indicator anyway beacuse onBarUpdate has already executed.

Would I not have to call again onBarupdate from onMarketData? Is that even possible?
Which method would you update the plots in?

[cls71]

I will not be able to update the indicator anyway beacuse onBarUpdate has already executed.

But you're not missing any data for the bar just completed. You have nothing to update from the OnMarketData. The trades (Last type) that will be received in the OnMarketData already will belong to the new bar.

Would I not have to call again onBarupdate from onMarketData? Is that even possible?
Which method would you update the plots in?

The plots should update them in the OnBarUpdate.
If you want to update the graphical look of your indicator within the OnMarketData you should override the OnRender method.
(Or save the changes you want to apply, and make them when you run the OnBarUpdate).

[joanrocagas]

But you're not missing any data for the bar just completed. You have nothing to update from the OnMarketData. The trades (Last type) that will be received in the OnMarketData already will belong to the new bar.

You don't miss any data but the indicator will update always one tick late. It will lag one trade tick behing the real market.

Flow of the events:

1. Trade tick is received. Maybe a 100 contracts trade.
2. onBarUpdate fires. isFirstTickOfBar is true. intrabar calculations are empty because onMarketData has not been fired yet.
3. onMarketData is fired. Intrabar calculations are done here. Maybe some private fields are updated. Indicator is lagging behind here, because is not updated. OnBarUpdate was fired before, so it didn't have a chance to update Value.
4. Some seconds with no trades. Indicator lagging, still at zero.
5.Trade tick is received.
6. onBarupdate is fired. Indicator updates with the intrabar calculations but again it lags behind one tick since onMarketData has not been called yet for the tick received on step 5.
...

See what I mean?

[cls71]

You don't miss any data but the indicator will update always one tick late. It will lag one trade tick behing the real market.

Flow of the events:

1. Trade tick is received. Maybe a 100 contracts trade.
2. onBarUpdate fires. isFirstTickOfBar is true. intrabar calculations are empty because onMarketData has not been fired yet.
3. onMarketData is fired. Intrabar calculations are done here. Maybe some private fields are updated. Indicator is lagging behind here, because is not updated. OnBarUpdate was fired before, so it didn't have a chance to update Value.
4. Some seconds with no trades. Indicator lagging, still at zero.
5.Trade tick is received.
6. onBarupdate is fired. Indicator updates with the intrabar calculations but again it lags behind one tick since onMarketData has not been called yet for the tick received on step 5.
...

See what I mean?

Yes, override OnRender. Works perfect without lag.

Other option very much easy (but I didnt test it) is recalc Plots object into OnMarketData and call ForceRefresh method.

[joanrocagas]

My main concern is not really the plots. I could live with the plots lagging behind a bit

My main concern is that if the indicators lag behind in the onBarUpdate method then the strategies will lag too, and the whole system is lagging one trade tick all the time.

My understanding is that strategies will check conditions based on the indicators Values. If those indicators Values are not updated tick by tick correctly, then the strategies are taking decisions always one tick late.

[cls71]

My main concern is not really the plots. I could live with the plots lagging behind a bit

My main concern is that if the indicators lag behind in the onBarUpdate method then the strategies will lag too, and the whole system is lagging one trade tick all the time.

My understanding is that strategies will check conditions based on the indicators Values. If those indicators Values are not updated tick by tick correctly, then the strategies are taking decisions always one tick late.

If you want to develop a strategy it includes all the logic about order-flow within the strategy (without calls to an external indicator) and launch the orders in the OnMarketData of the strategy. Zero lag.

If you want to develop an indicator to use it in strategies, save the data you want to share with the outside in the indicator's public properties. Although here you may find serious problems with multi-thread synchronization (for example, first is executed the OnMarketData of the strategy when the OMD of the indicator not yet).
You would have to use locks, etc. One possible solution will be to use custom events to publish the signals from the indicator to strategies.

The best solution is to include the operational method in the strategy.

[joanrocagas]

My idea is to develop some custom indicators first. Use the NT interface to plot them, see how they behave, etc... After that I want to build strategies using some of those indicators, maybe I'll include some other popular indicators, etc...

What you say it's an interesting idea but I don't want to have the whole logic inside only one method, the onMarketData of one strategy object. And I think that is not the purpose of NT.

I think I could take advantage of the granularity that NT offers splitting up the different parts into Indicators and Strategies.

This pluggable architecture is really great and allows it to see what parts works better and what parts need refining and so on.

I started to code my system in Java a while ago and I have the whole system advance tick by tick. It's also multithreaded but the components have to be synced tick by tick. Otherwise is just a mess of wrong calculations.

NT seems very good fit for my purpose and I think it can speed up a lot my development time.

I think NT should fix this or give some good indication on how to take advantage of all the features it offers in the way they are intended to do so. In my case, it's the Calculate onEachTick feature, combining Indicators, Strategies and onMarketData, onMarketDepth and onBarUpdate methods.

My tests tell me that these methods are not executed in the correct order. They even say that in the docs.

[joanrocagas]

Just to make this thread easier to follow.
This issue has been registered as a bug by the NT Team

http://ninjatrader.com/support/forum/showthread.php?t=94640

[NinjaTrader_Jim]

Hello,

We have reflected our post to reflect the current Help Guide.

In NinjaTrader 8 OnBarUpdate gets called before OnMarketData.

In NinjaTrader 7, there is no guaranteed ordering between OnBarUpdate and OnMarketData.