topic Re: Line of fit and intercept column formula, help with an error in Discussions

Line of fit and intercept column formula, help with an error

GRoberts86 — Tue, 03 Dec 2024 17:34:53 GMT

JMP18.1.1

I'm working with measurement data, where I'd like to compute a line of fit at a particular datapoint, using the neighbouring datapoints. To then obtain an x-axis intercept for y=0.

Looking around and trying to solve this myself, it seems I can do it using either the 'Linear Regression' or 'Least Squares Solve' function in JMP..

I've done this in a column formula (with an if statement to identify the datapoint about which to evaluate):

x = J( 2, 1, 0 );
y = J( 2, 1, 0 );
x[1, 1] = Lag( :y_column, 1 );
x[2, 1] = Lag( :y_column, -1 );
y[1, 1] = Lag( :x_column, 1 );
y[2, 1] = Lag( :x_column, -1 );
Coeffs = Linear Regression( y, x )[1];
Coeffs[1];

This code produces the right output, but when it runs it throws up an error:

'Unable to estimate the variance of parameter estimates because the rank of X equals the number of rows of X'

I don't really understand what the problem here is. I only want the intercept anyway. Is there a simple way to fix this? Or turn off the warning?

I can try a similar approach with the Least Squares function, but that throws up the same error.

I've made an example table which shows the problem. Rerun the formula on that table to see the error.

Thanks

Re: Line of fit and intercept column formula, help with an error

GRoberts86 — Tue, 03 Dec 2024 17:46:29 GMT

I seem to have immediately fixed this myself by increasing the number of datapoints in the regression to 3, including the current datapoint.

	x = J( 3, 1, 0 );
	y = J( 3, 1, 0 );
	x[1, 1] = Lag( :y_column, 1 );
	x[2, 1] = :y_column;
	x[3, 1] = Lag( :y_column, -1 );
	y[1, 1] = Lag( :x_column, 1 );
	y[2, 1] = :x_column;
	y[3, 1] = Lag( :x_column, -1 );
	Coeffs = Linear Regression( y, x )[1];
	Coeffs[1];

This stops the error coming up, which may be ok for now. I would still like to possibly calculate intercepts from only 2 points in future though. So my question still stands.

Thanks

Re: Line of fit and intercept column formula, help with an error

jthi — Tue, 03 Dec 2024 18:00:01 GMT

You could include the point at which you are doing the calculation at (so 3 points instead of two)

If(Row() > 1 & Row() < Col Number(:y_column),
	Linear Regression(:y_column[Index(Row() - 1, Row() + 1)]`, :x_column[Index(Row() - 1, Row() + 1)]`)[1][1];
,
	.
);

Re: Line of fit and intercept column formula, help with an error

GRoberts86 — Wed, 04 Dec 2024 12:54:52 GMT

I now have a problem with the intercept value returned not being correct...

I'll have to show an excerpt of real data to show this. File attached.

I'm evaluating the intercept at the row where another column is at it's maximum value.

The script is taking the values correctly from the adjacent rows, I can see in the log output that it is forming the matrix correctly.

It gives an output value for the intercept of '-1053478758'. But if I take those x/y values manually into a separate table and run a bivariate line fit on them, it gives an intercept of '-1.22', which is correct.

Code:

If( :"dlogIds/dVgs"n == Col Maximum( :"dlogIds/dVgs"n ),
	x = J( 3, 1, 0 );
	y = J( 3, 1, 0 );
	x[1, 1] = Lag( :I_Drain Abs, 1 );
	x[2, 1] = :I_Drain Abs;
	x[3, 1] = Lag( :I_Drain Abs, -1 );
	y[1, 1] = Lag( :V_GATE, 1 );
	y[2, 1] = :V_GATE;
	y[3, 1] = Lag( :V_GATE, -1 );
	Show( x );
	Show( y );
	Coeffs = Linear Regression( y, x )[1];
	Coeffs[1];
)

Can't figure out at all why it's not right....

Re: Line of fit and intercept column formula, help with an error

jthi — Wed, 04 Dec 2024 18:44:20 GMT

Could be some accuracy thing/oversight/bug. You could contact JMP support @Mark_Zwald

Names Default To Here(1);

ys = [-0.999986, -1.49994, -1.99997];
xs = [1.3500e-12, 1.0600e-10, 1.9700e-9];
xs2 = Matrix(Eval List({1.3500e-12*10^12, 1.0600e-10*10^12, 1.9700e-9*10^12}));

ls = Linear Regression(ys, xs, << print to log);
ls2 = Linear Regression(ys, xs2, << print to log);


dt = New Table("",
	New Column("Y", Numeric, Continuous, Values(ys)),
	New Column("X1", Numeric, Continuous, Values(xs)),
	New Column("X2", Numeric, Continuous, Values(xs2)),
);

biv1 = dt << Bivariate(Y(:Y), X(:X1), Fit Line({Line Color({230, 159, 0})}));
biv_estimates1 = Report(biv1)[Outline Box("Parameter Estimates"), Number Col Box("Estimate")];
Show(biv_estimates1 << get);

biv2 = dt << Bivariate(Y(:Y), X(:X2), Fit Line({Line Color({230, 159, 0})}));
biv_estimates2 = Report(biv2)[Outline Box("Parameter Estimates"), Number Col Box("Estimate")];
Show(biv_estimates2 << get);

Write();

Re: Line of fit and intercept column formula, help with an error

GRoberts86 — Thu, 05 Dec 2024 08:37:10 GMT

I was wondering if it was a precision issue. Looks like something in that function isn't handling the input with enough significant figures.

It works for now to use your fix and multiply the x axis values to something it can handle.

Thanks!

Re: Line of fit and intercept column formula, help with an error

MRB3855 — Thu, 05 Dec 2024 13:35:12 GMT

Hi @GRoberts86 : For completeness, the "error" message 'Unable to estimate the variance of parameter estimates because the rank of X equals the number of rows of X' is because you only have two points, and you are connecting them with a straight line. Therefore, there is no variability around the line...i.e., you aren't really carrying out a statistical procedure, you are just mathematically connecting two points. However, you using a statistical package to do that, via matrix algebra as shown here.

https://www.stat.cmu.edu/~cshalizi/mreg/15/lectures/13/lecture-13.pdf.

The statistical procedure you called (Linear Regression( y, x )[1]) attempts to do much more than just mathematically connect two points. It also provides confidence intervals, p-values, etc. for the slope and intercept. But in your case, since you only have two points (rank of X =2), there is no variability and hence no confidence intervals etc.

So, long story short, the "error" is really just a notification that there will be no statistical (as different from mathematical) output.

Re: Line of fit and intercept column formula, help with an error

GRoberts86 — Thu, 05 Dec 2024 14:33:26 GMT

@MRB3855

I realised that after thinking about it for a bit. If you're only working with 2 points, then good old 'equation of a line' y=mc+c is the appropriate thing to be using.

Still, a 'silent' flag would be nice haha.