topic Re: Difference of lines in Discussions

Difference of lines

BHarris — Sun, 11 Jun 2023 11:17:21 GMT

Suppose I have two lines, distance vs. time for two different moving objects. The time measurements for object 1 and object 2 are at arbitrary times, i.e., the measured times for object 1 aren't the same times as provided for object 2.

Is there a way in Graph Builder to plot the *difference* between those curves vs. time?

Re: Difference of lines

Mark_Bailey — Thu, 09 Sep 2021 19:33:36 GMT

The only way that you can compare to curves observed at different times is to model each one and compare the models at selected times. The model could be as simple as a linear interpolation as complex as any model type in JMP. Do you have JMP Pro? If so, you can treat the data as 'functions' and use the Functional Data Explorer.

I won't go into detail until I hear from you that the general approach makes sense.

Re: Difference of lines

BHarris — Thu, 09 Sep 2021 22:22:12 GMT

No, we just have the basic JMP v16.

Re: Difference of lines

Craige_Hales — Fri, 10 Sep 2021 13:56:53 GMT

If you had a pair of data tables, one per object, with columns for time and distance, you could save the formulas that would predict positions at between times. Subtracting the formulas at a Time would give an answer. I think Mark's question is if you think a linear/quadratic/other model from Fit Model would give a good-enough estimate of a object position, between measurements.

Re: Difference of lines

Mark_Bailey — Fri, 10 Sep 2021 14:31:36 GMT

Exactly. The complexity of the curve or shape will dictate the complexity of the model that is necessary. Can you show a picture of a plot of the data? Perhaps there is a theoretical model for such data?

We cannot help you further because the next step is the modeling, but we have to know which model in order to help you.

Re: Difference of lines

BHarris — Fri, 10 Sep 2021 21:51:19 GMT

@Mark_Bailey wrote:
Exactly. The complexity of the curve or shape will dictate the complexity of the model that is necessary. Can you show a picture of a plot of the data? Perhaps there is a theoretical model for such data?

We cannot help you further because the next step is the modeling, but we have to know which model in order to help you.

@Mark_Bailey , @Craige_Hales -- Yes, a linear fit between points would be fine. I realize the difference will be somewhat bouncy due to the linear interpolation, but that's fine, we're just looking at rough magnitudes.

Re: Difference of lines

txnelson — Fri, 10 Sep 2021 22:20:44 GMT

@Mark_Bailey If the two lines are linear, then a Fit Model of Height predicted by Weight, for the 2 sex groups end up having an term estimate of -2.079223. Wouldn't that be the difference between the lines when simplified to a linear regression?

Re: Difference of lines

Craige_Hales — Fri, 10 Sep 2021 23:09:12 GMT

I think what you want is called a piece-wise linear interpolation. I'm not sure what platform might produce a formula for that, but you can do something similar like this. Do your own testing if you use this!

dt = Open( "$desktop/ab.jmp" ); // table with unaligned times for a and b measurements
dt << sort( by( time ), replacetable( 1 ) ); // make sure time is ascending
aRows = dt << getrowswhere( SourceTable == "a" ); // get all the a
bRows = dt << getrowswhere( SourceTable == "b" ); // and b row indexes
aTimes = dt:time[aRows]; // get all the a and b times and positions in
aPositions = dt:position[aRows]; // matrices for the interpolate function
bTimes = dt:time[bRows];
bPositions = dt:position[bRows];

apos = Function( {time}, // tiny wrappers for interpolate
	Interpolate( time, aTimes, aPositions )
);
bpos = Function( {time},
	Interpolate( time, bTimes, bPositions )
);

dtnew = New Table( "new_ab", // new table is similar to old table
	New Column( "Source Table", Character( 1 ), "Nominal" ),
	New Column( "time", Numeric, "Continuous", Format( "Best", 12 ) ),
	New Column( "position", Numeric, "Continuous", Format( "Best", 12 ) )
);

bigdif = 0; // track the largest difference so it can have a ref line
bigtime = .;
// make a new table covering the range of the data
For( i = min(min(aTimes),min(bTimes))-10, i <= max(max(aTimes),max(bTimes))+10, i += 1,
	dtnew << addrows( 1 ); // interpolate a at time i
	dtnew:SourceTable = "a";
	dtnew:time = i;
	dtnew:position = apos( i );
	dtnew << addrows( 1 ); // interpolate b at time i
	dtnew:SourceTable = "b";
	dtnew:time = i;
	dtnew:position = bpos( i );
	dtnew << addrows( 1 ); // difference at time i
	dtnew:SourceTable = "diff";
	dtnew:time = i;
	dtnew:position = apos( i ) - bpos( i );
	If( Abs( dtnew:position ) > bigdif, // capture biggest diff
		bigdif = Abs( dtnew:position );
		bigtime = i;
	);
);

dtnew << Graph Builder(
	Size( 518, 448 ),
	Show Control Panel( 0 ),
	Variables( X( :time ), Y( :position ), Overlay( :Source Table ) ),
	Elements( Points( X, Y, Legend( 21 ) ), Smoother( X, Y, Legend( 22 ), Lambda( 0.000001 ) ) ),
	SendToReport(
		Dispatch( {}, "time", ScaleBox, {Add Ref Line( bigtime, "Solid", "Black", "big=" || Char( bigtime ), 1 )} ),
		Dispatch( {}, "Graph Builder", FrameBox, {Marker Size( 2 )} )
	)
);

Input data like this

And the interpolated data like this

Producing a graph like this:

The green curve is blue - red; the ref line is at the first biggest difference.

If this was a race, a won in 71 seconds and b lost in 99 seconds. a was furthest ahead at 43 seconds (well, maybe. You can't be too sure about b's position at time 43 in the original data.)

You can't get that from a simple linear fit on the original data:

Original data with linear fit.

Re: Difference of lines

BHarris — Mon, 13 Sep 2021 13:02:26 GMT

Wow, impressive solution! And very well documented, thank you!!

The only thing better would be if JMP just did this on its own without the intermediate scripting step, but aside from shift-clicking two lines in Graph Builder and asking for a difference line (which is conceptually messy from a mental-model-of-what-it's-doing standpoint), I can't really imagine how that would work.

Thanks, Craige!

(ps. Are you really retired? Doing JMP support for fun?)

Re: Difference of lines

Craige_Hales — Mon, 13 Sep 2021 13:14:42 GMT

Yes, retired, and I learn a lot answering questions.

@XanGreggfor the graph builder suggestion.

Re: Difference of lines

XanGregg — Mon, 13 Sep 2021 13:59:43 GMT

Here's a low scripting alternative using the Save Formula command available in the red triangle menu for a Smoother or Line of Fit element.

Starting with Craige's ab.jmp table using Overlay to get two lines/curves, use Save Formula to get a new column with a formula that looks like:

If( :Source Table == "a", Spline Eval( ... ), :Source Table == "b", Spline Eval( ... ), . ))

Go into the Formula Editor for the new column and change the formula to subtract the two Spline expressions:

Spline Eval( ... ) - Spline Eval( ... )

And use that as the Y in GB.

Re: Difference of lines

BHarris — Wed, 27 Oct 2021 15:04:31 GMT

Due to some bizarre licensing problems* I was only finally able to try this today. It works quite well, very elegant!

Two questions:

Is JSL powerful enough to make that formula change for me? So that whole process could be scripted into a .jsl script?
I expected the difference curve to update as I changed the lambda on the smoother but it doesn't. Looking deeper I see that the "Save formula" is actually saving off the spline parameters, not maintaining a connection back to the actual splines being used. I presume the latter is not reasonably possible?

Thanks again!

****************************************

* I was finally forced to update my Mac to 11.6, not realizing that JMP v14 wouldn't work on it. So I requested v16 through my company, and it literally took months after getting a PO to actually get a v16 license for me. There was tons of back-and-forth with tax certs and address changes and PO problems. I can't tell if the problem was my company's supply-chain or JMP/SAS's sales department, but getting a license if you have money in hand really shouldn't be that hard.

Re: Difference of lines

hogi — Sat, 28 Oct 2023 17:33:20 GMT

As a combination of the different suggestions, you could also use a simple interpolate() - interpolate() to calculate the difference and then use the lambda setting of the smoother plot to tune the granularity of the curve.

Open( "$DOWNLOADS/ab.jmp" );
New Column( "diff",
	Formula(
		rowsA = As Constant( Current Data Table() << get rows where( :Source Table == "a" ) );
		rowsB = As Constant( Current Data Table() << get rows where( :Source Table == "b" ) );
		xA = As Constant( :time[rowsA] );
		xB = As Constant( :time[rowsB] );
		posA = As Constant( :position[rowsA] );
		posB = As Constant( :position[rowsB] );
		If( Min( xA ) <= :time <= Max( xA ) & Min( xB ) <= :time <= Max( xB ),
			Interpolate( :time, xA, posA ) - Interpolate( :time, xB, posB ),
			.
		);
	)
);
Graph Builder(
	Variables( X( :time ), Y( :diff ), Y( :position ), Overlay( :Source Table ) ),
	Elements( Position( 1, 1 ), Points( X, Y, Overlay( 0 ), Legend( 1 ) ), Smoother( X, Y, Overlay( 0 ), Lambda( 0.02 ) ) ),
	Elements( Position( 1, 2 ), Points( X, Y ), Smoother( X, Y ) )
);