Bug report!

[aytacasan]

Hello,

First of all, i want to say that i'm using NT 7.0.1000.22 version. I didnt follow your newer versions for a while.

Today i tried to develop linear regression line indicator for MetaTrader4 platform. Before that i search from google that existing indicators and didnt like anyone (i'm a little bit rigorous about programming). Anyway i try to understand linear regression formula from wiki and khanacademy.org and at the last i wrote my own regression line indicator.

But after that i want to be sure its working correctly or not and i remember NT has LinReg indicator!!! God! I created new indicator on MetaTrader4 exactly the same as NT and only convert NT c# code to MQL language.

Result is... there is a little bit diffrence! I said myself that NT can not be wrong mine indicator must be wrong. But after a little work i found source of difference. And i'm sure (very likely) NT has a bug about its LinReg indicator.

My opinion for fix the bug : change LinReg source 46. row from "2 * Period - 1" to "2 * (Period - 1)"

But I want to be sure about that please inform me, am i right? or maybe you fixed it subsequent versions.

Best Regards,
Aytaç

[arbuthnot]

Very interesting point, Aytaç.

I've had a very brief look at the NT formula and I think it's correct.

I haven't had time to check it out properly in any way, but I'll be fascinated to read NT's reply.

[NinjaTrader_PatrickH]

Hello Aytaç,

Thank you for your post.

There has been no change to the code for the Linear Regression in NinjaTrader from version 22 to 26 (26 being the current).

To classify this as a "bug", what values are produced from your calculation? Why are these values true and not the values produced from NinjaTrader's calculation?

Your divisor would produce -4160 using 2 * (Period -1) if my period was 16, where NinjaTrader produces -5440. What value do you expect from the divisor and what period are you using (number of bars)? Why would we need to subtract 1 from our number of bars before multiplying?

Or are you looking for a line by line - cell by cell explanation of the code used in NinjaTrader?

[arbuthnot]

Hello Aytaç,

Thank you for your post.

There has been no change to the code for the Linear Regression in NinjaTrader from version 22 to 26 (26 being the current).

To classify this as a "bug", what values are produced from your calculation? Why are these values true and not the values produced from NinjaTrader's calculation?

Your divisor would produce -4160 using 2 * (Period -1) if my period was 16, where NinjaTrader produces -5440. What value do you expect from the divisor and what period are you using (number of bars)? Why would we need to subtract 1 from our number of bars before multiplying?

Or are you looking for a line by line - cell by cell explanation of the code used in NinjaTrader?

Thanks, Patrick.

There's a very good way to test whether an indicator formula is correct or not: plot it on a 1 Renko chart with the input series set to Median, period = 1

Please see attached image and you'll see the LinReg passes right through the middle of the bars every time.

The chances of this happening if the formula were wrong are billions to one against - if not impossible.

Aytaç, I hope all goes well with your trading with Ninja!

P.S. I don't think Aytaç meant there was an actual 'bug' in the system, other than an incorrect formula.

[aytacasan]

Thanks for your response my friend(arbuthnot),

I want to say that i have very different functions (but doing same thing) that calculating linear regression curve. One example of difference about codes;

For calculation linear regression you have to find sum of x values for a period.

NT doing like this;
>>>double sumX = (double) Period * (Period - 1) * 0.5;
Mine doing like this;
>>>for (int x=0; x<Period; x++) { sumx += x; }

Result is same, approach is different.

Codes are produce same results but calculations very different. One is mine other NT's after change NT's source code like my previous post they are generating same results This may be a coincidence? I dont think so. I'm looking forward to the answers.

Best Regards,
Aytaç

[aytacasan]

Patrick,

Sorry about previous post.
When i send it i didn't see your last response.

I don't know how can i prove it but...

if u want i can share my mql source code here and we can talk about it very clearly
or if u want i can write NT indicator that convert my MQL code to c#.

Which is acceptable for u?

PS : Also i want to correct one misunderstanding. I didn't say we have to subtract 1 from period. Only said there is a problem because of priority of c# mathematical operations;

2 * period - 1 not equel 2 * (period - 1)

Thanks,
Aytaç

[aytacasan]

arbuthnot,

Thanks for your correction; yes this is actually not a bug only wrong calculation about one indicator.
And also because of my English is not good, i don't understand u clearly your test method about the problem. Anyway i'll try test it like your said...

Thanks,
Aytaç

[NinjaTrader_Bertrand]

Aytac, could you perhaps attach your version so we could compare?

What I see for the divisor part in the LinReg and LinRegSlope calcs is this line -

double divisor = sumX * sumX - (double) Period * Period * (Period - 1) * (2 * Period - 1) / 6;

As far as I'm aware this is in line with other trading platforms implementations for this study.

[aytacasan]

Hi Bertrand,

If i understand you correctly, my opinion is divisor code line must be like this;

double divisor = sumX * sumX - (double) Period * Period * (Period - 1) * (2 * (Period - 1)) / 6;

If i misunderstood u please inform me...

PS : Also i have to say that i'm not comparing platforms, already MT4 not containing linear regression indicator because it's not rich about indicators like NT. I wrote my own custom indicator. Actually i dont know what is that mean divisor etc... in your lin reg indicator so i decide to post my indicator core function. I'm sure your developer team can be read easily the code because MQL syntax very similar c++ like c#.

int OnCalculate(const int rates_total,
const int prev_calculated,
const datetime &time[],
const double &open[],
const double &high[],
const double &low[],
const double &close[],
const long &tick_volume[],
const long &volume[],
const int &spread[])
{
if (rates_total < Period || Period < 1)
return(0);
//---
int limit = rates_total - prev_calculated;
if (prev_calculated > 0)
limit++;
else
limit-=Period-1;
//---
for (int y=limit-1; y>=0; y--)
{
double sumx = 0;
double sumy = 0;
double sumx2 = 0;
double sumy2 = 0;
double sumxy = 0;
//---
for (int x=0; x<Period; x++)
{
sumx += x;
sumy += close[x + y];
sumx2 += x * x;
sumy2 += y * y;
sumxy += x * close[x + y];
}
double m = (Period * sumxy - sumx * sumy) / (Period * sumx2 - sumx * sumx);
double r = (Period * sumxy - sumx * sumy) / MathSqrt((Period * sumx2 - sumx * sumx) * (Period * sumy2 - sumy * sumy));
//---
Buffer[y] = ((sumy + m * sumx) / Period) - m * Period;
}
//--- return value of prev_calculated for next call
return(rates_total);
}

Thanks,
Aytaç

[NinjaTrader_Bertrand]

Aytac, the posted line from my reply contains the correct working code formatting needed.

double divisor = sumX * sumX - (double) Period * Period * (Period - 1) * (2 * Period - 1) / 6;

This is the formula we would advice using, I compared for example to Tradestation implementation and it matches exactly if same length and data is used.

[arbuthnot]

Aytaç

Firstly, your English - as you write it - is really excellent !

Now, I demonstrated in post #4, that it is extremely unlikely NT's LinReg is coded incorrectly.

Please see attached image.

I basically re-coded LinReg with your suggestion: this is the changed code:

double	divisor = sumX * sumX - (double) Period * Period * (Period - 1) *[B] (2 * (Period - 1)[/B]) / 6;

The image shows that this is wrong!!! My first image shows NT's LinReg (which has been around for years and no one's complained before) must be right !

Forgive me, Aytaç, but it would have been a good idea to do what I've just done to test this before posting in the Forum.

If the brilliant Ninja staffers can help you in some way - and they're far more patient than I am - I'm sure they will!

I'm back to my own coding - and seriously, thanks, Aytaç, you really have done me a big favour for reminding me about LinReg. I'd forgotten about it!

[aytacasan]

My friend,

Please tell me why u are doing your tests on renko charts. What is the point?
I late response because after your post i decide to watch again all videos that related regression formulas from www.khanacademy.org.

Please if u have time watch it: https://www.khanacademy.org/math/pro...-and-causality

I still to try to solve why mine indicator wrong and nt's or trade stations's right... I posted my code before if u see any penalty please inform me.

Best Regards,
Aytaç

[aytacasan]

Hi again,

i convert my code from MQL to c#. I didn't write NT indicator for a long time so i'm not sure conversion is ok(i know there is more effective ways to write it for ninja but i try to maintain same logic). I'm really worry about that how this code produce same plot after change NT's linreg code like i said?

Actually i didn't worry NT's linreg indicator right or not. Because in my country there is no broker to serve NT platform for trading. So i can not use it for real account trade. But still i want to research this situation.

#region Using declarations
using System;
using System.ComponentModel;
using System.Diagnostics;
using System.Drawing;
using System.Drawing.Drawing2D;
using System.Xml.Serialization;
using NinjaTrader.Cbi;
using NinjaTrader.Data;
using NinjaTrader.Gui.Chart;
#endregion

// This namespace holds all indicators and is required. Do not change it.
namespace NinjaTrader.Indicator
{
/// <summary>
/// Enter the description of your new custom indicator here
/// </summary>
[Description("Enter the description of your new custom indicator here")]
public class LinRegEx : Indicator
{
#region Variables
// Wizard generated variables
private int period = 14; // Default setting for Period
// User defined variables (add any user defined variables below)
#endregion

/// <summary>
/// This method is used to configure the indicator and is called once before any bar data is loaded.
/// </summary>
protected override void Initialize()
{
Add(new Plot(Color.FromKnownColor(KnownColor.OrangeRed), PlotStyle.Line, "MyLinReg"));
Overlay = true;

}

/// <summary>
/// Called on each bar update event (incoming tick)
/// </summary>
protected override void OnBarUpdate()
{
if (CurrentBar < Period)
return;

double sumx = 0;
double sumy = 0;
double sumx2 = 0;
double sumy2 = 0;
double sumxy = 0;
for (int x=0; x<Period; x++)
{
sumx += x;
sumy += Close[x];
sumx2 += x * x;
sumy2 += (Bars.Count - (CurrentBar+1)) * (Bars.Count - (CurrentBar+1));
sumxy += x * Close[x];
}
double m = (Period * sumxy - sumx * sumy) / (Period * sumx2 - sumx * sumx);
double r = (Period * sumxy - sumx * sumy) / Math.Sqrt((Period * sumx2 - sumx * sumx) * (Period * sumy2 - sumy * sumy));

MyLinReg.Set(((sumy + m * sumx) / Period) - m * Period);
}

#region Properties
[Browsable(false)] // this line prevents the data series from being displayed in the indicator properties dialog, do not remove
[XmlIgnore()] // this line ensures that the indicator can be saved/recovered as part of a chart template, do not remove
public DataSeries MyLinReg
{
get { return Values[0]; }
}

[Description("")]
[GridCategory("Parameters")]
public int Period
{
get { return period; }
set { period = Math.Max(1, value); }
}
#endregion
}
}