Slow to Calculate

[mlprice12]

I am hoping the team could direct me and to help me understand why my custom indicator is slow to calculate.
It takes close to an hour for the calculation.

Amount of data = 1 day
Series is ES
Data type is 3 Tick
MaxBarsLookback is set to infinite

Thank you

[bltdavid]

I don't think anyone will be able to help you.

You've uploaded a compiled assembly.

A compiled assembly provides zero assistance to those trying
to understand the problems in your code -- no one can see it.

Want help?
Got source?
Trim(*) it.
Upload it.

* For large amounts of code, please reduce to least amount
of code that still demonstrates the problem.

With that said, have you added Print statements to identify where
your code is at? Are you re-calculating anything over and over that
should be calculated just once? Are you employing any big ass
crazy long loops (especially inner loops)?

[mlprice12]

Thanks bltdavid ,

I didn't realize I compiled it.
The code is pretty short so I'll clip it here.
This is my first time working with OnRender and for loops so I'm guessing the slowdown is occurring there.
I've used Prints and every portion of the code fires. I'll dig deeper into the prints to see which part is causing the slowdown.

 else if (State == State.Configure)
{

AddVolumetric("ES 03-22", BarsPeriodType.Tick,3, VolumetricDeltaType.BidAsk, 1);
}
else if (State == State.DataLoaded)
{

barVol = new Series<float>(BarsArray[1]);
sizeVol = new Series<float>(BarsArray[1]);
dPer = new Series<float>(BarsArray[1]);
mid = new Series<double>(BarsArray[1]);
}

}
private Series<float> barVol;
private Series<float> sizeVol;
private Series<float> dPer;
private Series<double> mid;


protected override void OnBarUpdate()
{
if (CurrentBars[1] < 1)
return;
if (Bars == null)
return;

NinjaTrader.NinjaScript.BarsTypes.VolumetricBarsTy pe barsType = BarsArray[1].BarsType as
NinjaTrader.NinjaScript.BarsTypes.VolumetricBarsTy pe;

if (barsType == null)
return;
if(BarsInProgress != 1)
return;

if (CurrentBars[1] > 1)
{

barVol[0] = barsType.Volumes[CurrentBars[1]].TotalVolume;
sizeVol[0] = Convert.ToSingle((barVol[0] / (barsType.Volumes[CurrentBars[1]].Trades))*1.0000);
dPer[0] = barsType.Volumes[CurrentBars[1]].BarDelta / barVol[0];
mid[0] = (Highs[1][0] + Lows[1][0])/2;

}
}

protected override void OnRender(ChartControl chartControl, ChartScale chartScale)
{
for (int idx = ChartBars.FromIndex; idx <= ChartBars.ToIndex; idx++)
{

if(mid.IsValidDataPointAt(idx))
{
float x = chartControl.GetXByBarIndex(ChartBars, idx);
float y = chartScale.GetYByValue(mid.GetValueAt(idx));

// create two vectors to position the ellipse
SharpDX.Vector2 startPoint = new SharpDX.Vector2(x, y);
SharpDX.Vector2 endPoint = new SharpDX.Vector2(x, y);

// calculate the center point of the ellipse from start/end points
SharpDX.Vector2 centerPoint = (startPoint + endPoint) / 2;

// set the radius of the ellipse
if(sizeVol.IsValidDataPointAt(idx))
{
float multi = Convert.ToSingle(1.755);
float radiusX = sizeVol.GetValueAt(idx)*multi;
float radiusY = sizeVol.GetValueAt(idx)*multi;

// construct the Ellipse struct to describe the position and size the drawing
SharpDX.Direct2D1.Ellipse ellipse = new SharpDX.Direct2D1.Ellipse(centerPoint, radiusX, radiusY);

// define the brush used in the rectangle
if(sizeVol.GetValueAt(idx)>=3)
{
if(dPer.GetValueAt(idx)>=.1)
{
SharpDX.Direct2D1.SolidColorBrush customDXBrush2 = new SharpDX.Direct2D1.SolidColorBrush(RenderTarget, new SharpDX.Color4(new SharpDX.Color3(0f, 0f, 255f), 0.490f));
RenderTarget.FillEllipse(ellipse, customDXBrush2);
customDXBrush2.Dispose();
}

else if(dPer.GetValueAt(idx)<=-.1)
{
SharpDX.Direct2D1.SolidColorBrush customDXBrush5 = new SharpDX.Direct2D1.SolidColorBrush(RenderTarget, new SharpDX.Color4(new SharpDX.Color3(255f, 0f, 0f), 0.490f));
RenderTarget.FillEllipse(ellipse, customDXBrush5);
customDXBrush5.Dispose();
}
else
{
SharpDX.Direct2D1.SolidColorBrush customDXBrush6 = new SharpDX.Direct2D1.SolidColorBrush(RenderTarget, new SharpDX.Color4(new SharpDX.Color3(200f, 200f, 200f), 0.490f));
RenderTarget.FillEllipse(ellipse, customDXBrush6);
customDXBrush6.Dispose();
}
}

[bltdavid]

What is the primary data series of the chart?
What is the Calculate setting?

Hmm, here's an idea...
Have you tried changing all Series<float> to Series<long>?

Why?
It might make better sense to use a series of the most fundamental
type (which looks to be long) and then cast to float for the SharpDX
routines when needed.

Why?
Your OnBarUpdate is being called (I presume) at OnBarClose for
every 3-Tick bar on your secondary data series -- so lots of casting
to store as float happens on every OnBarUpdate.

But, OnRender should be called a lot less (I think) since it would
only deal with drawing objects against the primary bars in the chart,
thus casting from long to float for SharpDX routines may be the
lesser performance hit.

Just an idea.
But, then again, it may not make any difference.
But, then again, seems like an easy test to find out.



PS: All this also depends on your primary data series and your
Calculate setting -- I mean, I presume your primary data series
is not a 3-Tick data series, and I presume Calculate setting is
set to OnBarClose.

[bltdavid]

I am hoping the team could direct me and to help me understand why my custom indicator is slow to calculate.
It takes close to an hour for the calculation.

Amount of data = 1 day
Series is ES
Data type is 3 Tick
MaxBarsLookback is set to infinite

How do you know it takes an hour for each calculation?
Can you be more specific how this was determined?
What are you seeing visually?

Have you removed the OnRender portion to isolate the
issue to your OnBarUpdate calculations?

[bltdavid]

Thanks bltdavid ,
The code is pretty short so I'll clip it here.

Yeah, your OnBarUpdate and OnRender are rather brief, and
those routines usually constitute the bulk of most indicators.

It would be useful to be able to reproduce exactly what you're
seeing, but one would first need a fully compilable piece of code,
along with the steps needed to reproduce (chart setup, etc).

Are you able to do that?
Can you attach the indicator's .cs file?

[bltdavid]

These lines of code are rather poor.

float multi = Convert.ToSingle(1.755);
float radiusX = sizeVol.GetValueAt(idx)*multi;
float radiusY = sizeVol.GetValueAt(idx)*multi;

should be,

const float multi = 1.755f;
float radiusX = sizeVol.GetValueAt(idx)*multi;
float radiusY = radiousX;

And if 'multi' is not reused,

float radiusX = sizeVol.GetValueAt(idx)*1.755f;
float radiusY = radiousX;

and since 'radiusX' and 'radiusY' are the same, just use,

float radius = sizeVol.GetValueAt(idx)*1.755f;

and use 'radius' everywhere instead.

[bltdavid]

Also, inside your OnRender, you have this loop,

for (int idx = ChartBars.FromIndex; idx <= ChartBars.ToIndex; idx++)

but that is for bar indexes on the primary data series, ya know, the
bars displayed on your chart. I'm pretty sure OnRender is never called
for any other bar series besides the primary data series.

Also, these Series are sync'ed to the secondary data series,

barVol = new Series<float>(BarsArray[1]);
sizeVol = new Series<float>(BarsArray[1]);
dPer = new Series<float>(BarsArray[1]);
mid = new Series<double>(BarsArray[1]);

which certainly is a valid thing to do, but the index values in the
OnRender loop are not really valid for these four series, I mean
the number of bars in the primary data series vs number of bars
in the secondary data series could be vastly different.

My point is, ChartBars.FromIndex/ToIndex don't really have
anything to do with the secondary data series, yet you're using
these bar indexes designed for your primary data series to access
values for series synced to secondary data series -- this seems
like a huge disconnect in your logic.

What is the primary data series of the chart?
Is it also 3-Tick?

[mlprice12]

Thanks so much for your help bltdavid
Here are the attached screenshot of instrument specs and an uncompiled zip of the indicator.

I'll dive into your questions and insights in a minute after I get some food in me.

I really appreciate it. You're a saint

[bltdavid]

Oh, ok ...

So, the primary and secondary data series are both 3-Tick.
That means my comments about the for loop in OnRender
can be ignored -- both should have same number of bars.

You must be aware that 3-Tick bar series on an ES chart
will produce a huge number of bars ...

Can you explain in detail what it is you're trying to achieve?

[bltdavid]

Why use a secondary data series for the Volumetric data series?

I mean, can't your indicator work if you choose the primary data
series bar type as 'Volumetric'? Isn't this available in your drop
down when you first setup the chart?

It seems like the 3-Tick Volumetric secondary data series is
simply not necessary -- just setup the initial chart as a 3-Tick
Volumetric data series in the new chart setup dialog.

Of course, when you setup your chart that way, you'd have to
change your indicator code by omitting usage of the secondary
data series -- most of your code actually becomes simpler,
since it reverts to normal usage of BarsArray[0] (which is the
default for most things anyways), because that is the primary
data series.

[It would take some effort to test this, so I'm not sure if you've tried
doing it that way. But, I presume you had, since the way you're
currently doing it is the harder, more complex way, and presumably,
you have a reason for using this approach.]

Does the simpler Volumetric as primary data series not work?

Just curious.

[NinjaTrader_ChelseaB]

Hello mlprice12,

I would check first if the logic in OnBarUpdate or the logic in OnRender() is causing the performance hit.

If you comment out all logic in OnRender() does the behavior continue?

In OnRender() I can see a lot of new objects being created. Re-using these may improve the performance.

https://ninjatrader.com/support/forum/forum/ninjatrader-8/indicator-development/98639-performance-optimization?p=789606#post789606

https://ninjatraderecosystem.com/user-app-share-download/dvaluearea/

With a 3-tick series, I would agree this would generate a lot of bars to process. Is the behavior the same on a 1 minute chart?

[mlprice12]

bltdavid NinjaTrader_ChelseaB
So the reason for the 3 tick chart is to render my ellipses as quick as possible because I'm scalping. Also I'm using the OrderFlowMarketDepthMap and I need it to update/render as fast as possible to visualize live changes in Bid/Ask volume.

Attached is a visualization

To be clear, each calculation doesn't take an hour but the initial calculating once the chart is opened takes close to an hour.
I realize there are a ton of bars but it is necessary for what I am trying to do.

Today I had the issue that after opening a trade the chart froze for 3 seconds and then the data and chart rendering continued to be delayed.

To be clear if I want to keep the ellipses printed the only Re-use will be with the brushes?

Thanks a lot

[NinjaTrader_ChelseaB]

Hello mlprice12,

That looks like it takes a lot of processing.

The brushes would be re-used if they are the same color.
The ellipse would also be re-used if its the same object, and the x and y coordinates can be modified.

[mlprice12]

NinjaTrader_ChelseaB

My CPU and Memory don't exceed 35% so is this a NinjaTrader platform problem with processing?