topic Re: Line Profile in JMP? in Discussions

Line Profile in JMP?

hogi — Sun, 11 Jun 2023 11:29:16 GMT

Is there a function available in JMP to generate a line profile from a Heatmap Plot, similar to "plot profile" in imageJ?
e.g. via hidden shift-rightClick menu of a line annotation?

Re: Line Profile in JMP?

pauldeen — Thu, 10 Nov 2022 08:38:48 GMT

So far the best i've been able to do is generate a sufficiently high density point cloud (grid in contour plot) and then use a graph builder with a local data filter to restrict the depth around the line you are interested in. It is not easy, so this would be good functionality to have built in!

Re: Line Profile in JMP?

hogi — Thu, 10 Nov 2022 15:43:39 GMT

OK, I will add a wish to implement it directly in JMP :)
edit: here it is

As a workaround:
@pauldeen Your suggestion is already quite good, but it would be great to have the chance to >draw< the line instead of defining it with a filter function.

Your suggestion helped me to get away from the "line annotation" idea.

How about using the lasso selection in the Heatmap Plot? (ifthe monitor allows "touch", one can draw a straight-ish line with the finger : )
Afterwards, the selection can be used to filter the data, average over the non-excluded values and generate the plot:

result:

Re: Line Profile in JMP?

hogi — Thu, 10 Nov 2022 16:57:15 GMT

Combining both ideas ...
For Line Annotations, what is the JSL command to get the start and end point (in the coordinate system of the plot)?

dt = Open( "$SAMPLE_STIPS/Wafer Stacked Small.jmp" );
gb = Graph Builder(
	Size( 534, 464 ),
	Show Control Panel( 0 ),
	Graph Spacing( 4 ),
	Variables( X( :X_Die ), Y( :Y_Die ), Color(:Defects) ),
	Elements( Heatmap( X, Y, Legend( 5 ) ) ),

);

myLine = gb << Add Line Annotation( Line( {150, 120}, {450, 400} ), Color( "Light Yellow" ), Thick( 1 ), Point to( 1 ) );
myLine << Get coordinates;

// gap in the code -------------------

x1 = 10;
x2 = -11;
y1 = -13;
y2 = 10;

dx = x2 - x1;
dy = y2 - y1;

xx = :x_Die << get values();
yy = :y_Die << get values();

px = (xx - x1);
py = (yy - y1);
dotp = (px * dx + py * dy);
denom = dx * dx + dy * dy;
proj = dotp :* dotp / denom;
lensq= px :* px + py :* py;
dist = Sqrt( lensq - proj );
selection = Loc( dist < 0.5 );
posAlongLine=sqrt(lensq[selection]);

dt << Select rows(selection);
subS= dt << Subset(Invisible, Selected Rows( 1 ), columns( :X_Die, :Y_Die, :Defects ), hidden );

subS << 
	New Column("posAlongLine", Numeric, "Continuous");
subS:posAlongLine << set values(posAlongLine);

subS << Graph Builder(
	Transform Column( "value", Formula( Col Mean( :Defects,:X_Die,Y_Die ) ) ),
	Size( 530, 357 ),
	Show Control Panel( 0 ),
	Graph Spacing( 4 ),
	Variables( X( :posAlongLine ), Y( :value ) ),
	Elements(
		Points( X, Y, Legend( 3 ) ),
		Smoother( X, Y, Legend( 4 ), Lambda( 0.5 ) )
	)
)

Re: Line Profile in JMP?

jthi — Thu, 10 Nov 2022 18:11:23 GMT

Using Mouse trap with some support variables to allow user to "drag" a line, the script is a mess but I think there could be something here after some cleaning and error handling.

Names Default To Here(1);
first = 1;
startx = 0;
starty = 0;
exx = 0;
exy = 0;
mouse_released = 0;

dt = Open("$SAMPLE_STIPS/Wafer Stacked Small.jmp");

xx = :x_Die << get values();
yy = :y_Die << get values();

New Window("",
	H List Box(
		gb = dt << Graph Builder(
			Size(534, 464),
			Show Control Panel(0),
			Graph Spacing(4),
			Variables(X(:X_Die), Y(:Y_Die), Color(:Defects)),
			Elements(Heatmap(X, Y, Legend(5)))
		),
		V List Box(
			Spacer Box(size(600, 0)),
			H List Box(
				Spacer box(size(0, 500)),
				gb2_container = V List Box();
			)
		)
	)
);

sel_rows = Expr(
	x1 = startx;
	x2 = exx;
	y1 = starty;
	y2 = exy;
	
	dx = x2 - x1;
	dy = y2 - y1;
	
	px = (xx - x1);
	py = (yy - y1);
	
	dotp = (px * dx + py * dy);
	denom = dx * dx + dy * dy;
	proj = dotp :* dotp / denom;
	lensq = px :* px + py :* py;
	dist = Sqrt(lensq - proj);
	selection = Loc(dist < 0.5);
	If(N Items(selection) > 0,
		posAlongLine = Sqrt(lensq[selection]);

		If(!IsEmpty(dt_temp),
			Close(dt_temp, no save);
		);
		
		dt << Clear Select;
		dt << Select rows(selection);
		dt_temp = dt << Subset(Invisible, Rows(selection), columns(:X_Die, :Y_Die, :Defects), hidden);
		dt_temp << New Column("posAlongLine", Numeric, "Continuous", values(posAlongLine));

		gb2_container << Append(
			gb2 = dt_temp << Graph Builder(
				Transform Column("value", Formula(Col Mean(:Defects, :X_Die, Y_Die))),
				Size(530, 357),
				Show Control Panel(0),
				Graph Spacing(4),
				Variables(X(:posAlongLine), Y(:value)),
				Elements(Points(X, Y, Legend(3)), Smoother(X, Y, Legend(4), Lambda(0.5)))
			);
		);
	);
);

l = Report(gb)[FrameBox(1)] << add graphics script(
	Mousetrap(
		If(first == 1,
			dt << Clear Select;
			startx = x;
			starty = y;
			first++;
		);
		exx = x;
		exy = y;,
		first = 1;
		mouse_released = 1;
	);
	
	myline = Line({startx, starty}, {exx, exy});
	
	If(mouse_released,
		sel_rows;
	);
	mouse_released = 0;

);

Re: Line Profile in JMP?

hogi — Thu, 10 Nov 2022 18:15:46 GMT

cool! :)

Re: Line Profile in JMP?

hogi — Thu, 10 Nov 2022 18:51:03 GMT

I wondered why there is no need for a

try(gb2<< delete;);

in front of the "Append" function.

Because gb2 is newly defined every time and this triggers the automatic deletion of the old plot?

~~(without "gb2 =" the plots are appended)~~ actually: no
wow! why?

It just doesn't work if the code is executed twice. Then a second linecut is Appended.

why?

Re: Line Profile in JMP?

jthi — Thu, 10 Nov 2022 18:45:07 GMT

I lazily let gb2 to be deleted when dt_temp is closed.

Re: Line Profile in JMP?

hogi — Sat, 12 Nov 2022 19:30:35 GMT

The Linecut adjusts now to the summary statistics (N, Mean, Median, Mode ...) of the Heatmap plot,

uses the X/Y group/wrap "column" as it's own overlay

and checks the global and local data filter

As input plot, you can use either the first/commented part - or your own Heatmap plot ...

	
/*dt = Open( "$SAMPLE_STIPS/Wafer Stacked Small.jmp" );


dt << Graph Builder(
	Size( 534, 464 ),
	Show Control Panel( 0 ),
	Graph Spacing( 4 ),
	Variables( X( :X_Die ), Y( :Y_Die ), Wrap( :Lot, Levels per Row( 3 ) ), Color( :Defects, Summary Statistic( "Sum" ) ) ),
	Elements( Heatmap( X, Y, Legend( 5 ) ) ),
	SendToReport(
		Dispatch( {}, "X_Die", ScaleBox, {Min( -22 ), Max( 22 ), Inc( 5 ), Minor Ticks( 4 )} ),
		Dispatch( {}, "Y_Die", ScaleBox, {Min( -20 ), Max( 22 ), Inc( 5 ), Minor Ticks( 4 )} ), 

	)

);*/


// second part --------------------------------------------------------

//Names Default To Here( 1 );

New Namespace(
	"LineScan"
);


LineScan:LS_Initialize = Function( {}, // variables used for communication between the Linescan and the Mousetrap
	LineScan:MyfirstClick = 1;
	LineScan:startx = 0;
	LineScan:starty = 0;
	LineScan:exx = 0;
	LineScan:exy = 0;
	LineScan:mouse_released = 0;
	LineScan:gb = 1;
	LineScan:dt = 1;
);

LineScan:CloseTemporaryTables() = Function( {},
	dtList = Get Data Table List();
	For Each( {checkDt, idx}, dtList,
		If( Not(IsMissing(Regex( checkDt << get name(), "tempDTLineCutXY.*" ))) ,
			Print( checkDt << get name() || ": close" );
			Close( checkDt, no save )
		)
	);
);

	
LineScan:LS_findVariables = Function( {},
	foundX = 0;
	foundY = 0;
	foundCol = 0;
	allVariables = (LineScan:gb << get variables);
	For Each( {var, idx}, allVariables,
		If( Arg( var[2] ) == "X",
			xVar = var[1];
			foundX = 1;
		);
		If( Arg( var[2] ) == "Y",
			yVar = var[1];
			foundY = 1;
		);
		If( Arg( var[2] ) == "Color",
			colVar = var[1];
			foundCol = 1;
		
		);
	);
		
	LineScan:myStatistics = "Mean";
	LineScan:myGroups = {};
	gbScript = LineScan:gb << Get Script;
	numArgs = N Arg( gbScript );
	For( i = 1, i <= numArgs, i++,
		If( Head Name( Arg( gbScript, i ) ) == "Variables", 
			//Print( "found variables", i );
			nVariables = N Arg( Arg( gbScript, i ) );
			For( k = 1, k <= nVariables, k++,
				If(
					Head Name( Arg( Arg( gbScript, i ), k ) ) == "Color", 
						//Print( "found color", k );
						nEntries = N Arg( Arg( gbScript, i ) );
						For( m = 1, m <= nEntries, m++,
							If( Head Name( Arg( Arg( Arg( gbScript, i ), k ), m ) ) == "Summary Statistic", 
								//Print( "found stat", m );
								LineScan:myStatistics = Arg( Arg( Arg( Arg( gbScript, i ), k ), m ), 1 )
							)
						);,
					myHead = Head Name( Arg( Arg( gbScript, i ), k ) );
					Contains( {"Group X", "Group Y", "Wrap"}, myHead );, 
						//Print( "found group ", k );
						Insert Into( LineScan:myGroups, Arg( Arg( Arg( gbScript, i ), k ), 1 ) )
				), 

			);
		)
	);
	foundX * foundY * foundCol;
);

		
LineScan:LS_sel_rows = Function( {}, 
		
	failed = 0;
	xx = xVar << get values();
	yy = yVar << get values();
	nr = N Items( yy );


	x1 = LineScan:startx;
	x2 = LineScan:exx;
	y1 = LineScan:starty;
	y2 = LineScan:exy;
	
	dx = x2 - x1;
	dy = y2 - y1;
	d1_2sq=( dx * dx + dy * dy );
	d1_2=  Sqrt(d1_2sq);// distance between points
			
			
		
	If( And( dx == 0, dy == 0 ),
		Print( "no line" );
		failed = 1;
	, 
	
		px1 = (xx - x1);
		py1 = (yy - y1);
	
		dotp1 = (px1 * dx + py1 * dy);
		proj1sq = dotp1 :* dotp1 / d1_2sq; // projection along the line
		proj1=sqrt(proj1sq);
		lensq = px1 :* px1 + py1 :* py1; // distance to pt1
		dist = Sqrt( lensq - proj1sq ); //Pythagoras
		
		selection = J( nr, 1 );
		width = If( Is Empty( widthEB ),
			Print( "use default" );
			0.7;
		,
			widthEB << get()
		);
		selection[Loc( dist > width )] = .;
		
		px2 = (xx - x2);
		py2 = (yy - y2);
		dotp2 = (px2 * dx + py2 * dy);
		proj2sq = dotp2 :* dotp2 / d1_2sq;
		proj2=sqrt(proj2sq);

		
		selection[Loc( proj1 > d1_2 )] = .; //remove points too far away (radius aound start and end)
		selection[Loc( proj2 > d1_2 )] = .;
		
		filteredRows = Loc((1::nr)`);
		Try(
			ldf = Linescan:TheReport["Local Data Filter"] << get scriptable object(); // wonderful (thank you Jim)
			filteredRows = ldf << Get Filtered Rows;
		);
		
		
		excludedRows = J( nr, 1 );
		If(!IsMissing(filteredRows),	
		excludedRows[filteredRows] = .;
		excludedRows=Loc(excludedRows);
		selection[excludedRows] = .; //apply data filter
		);
		selection[LineScan:dt << Get Excluded Rows()] = .; // apply global data filter
		
		
		selection = Loc( selection ); //just keep the non-empty ones
		If( N Items( selection ) <= 0,
			Print( "nothing selected" );
			failed = 1;
		,
			posAlongLine = proj1[selection];

			LineScan:CloseTemporaryTables();
				

			LineScan:dt << Clear Select;
			LineScan:dt << Select rows( selection );

					
			LineScan:dt_tempLineCut = LineScan:dt << Subset( Invisible, Rows( selection ), Selected columns only( 0 ), hidden );
			LineScan:dt_tempLineCut << New Column( "position along line", Numeric, "Continuous", values( posAlongLine ) );
			LineScan:dt_tempLineCut << Set Name( "tempDTLineCutXY" );
		);
	);
	Not( failed );
);
			
LineScan:GetStatisticsFunction = Function( {myStat},
	Print( "implement" )
);

LineScan:LS_prepareExpressions = Function( {myStat}, 
	//Print( "LS: prepareExpressions" );
	myStatisticsFunction = Match( myStat,
		"N", Name Expr( Col Number ),
		"Mean", Name Expr( Col Mean ),
		"Mode", Name Expr( Col Mode ),
		"Sum", Name Expr( Col Sum ),
		"Std Dev", Name Expr( Col Std Dev ),
		"Min", Name Expr( Col Minimum ),
		"Max", Name Expr( Col Maximum ),
		"Median", Name Expr( Col Median ),
		"Range", Name Expr( Col Range ),
		"Cumulative Sum", Name Expr( Col Cumulative Sum ),
		Name Expr( Col Number )
	);
	LineScan:YvariableName = myStat || "(" || (colVar << Get Name) || ")";
				
	LineScan:preparedYFunction = Substitute(
			Expr(
				__aggregation__( __colVar__, __xVar__, __yVar__ )
			),
		Expr( __aggregation__ ), Name Expr( myStatisticsFunction ),
		Expr( __colVar__ ), Name Expr( colVar ),
		Expr( __xVar__ ), Name Expr( xVar ),
		Expr( __yVar__ ), Name Expr( yVar ), 

	);
	If( myStat == "Median",
		LineScan:preparedYFunction = Insert( Name Expr( LineScan:preparedYFunction ), 0.5, 2 )
	);
	For Each( {GroupVar, idx}, LineScan:myGroups,
		LineScan:preparedYFunction = Insert( Name Expr( LineScan:preparedYFunction ), Name Expr( GroupVar ) )
	);
					
	LineScan:preparedVariables = Expr(
		Variables( X( :position along line ), Y( :YVariable ) )
	);
	If( N Items( LineScan:myGroups ),
		LineScan:preparedVariables = Insert( Name Expr( LineScan:preparedVariables ), Eval Expr( Overlay( Expr( LineScan:myGroups[1] ) ) ) )
	);
	1 //return value, don't delete !!!
	;
);

LineScan:LS_prepareWindow = Function( {}, 
	//Print( "LS: prepareWindow" );
	If( Is Empty( LineScan:LinecutWindow ),
		LineScan:LinecutWindow = New Window( "Linecut",
			Show Menu( 0 ),
			Show Toolbars( 0 ),
			V List Box(
				H List Box( Text Box( "width of the linecut: " ), widthEB = Number Edit Box( 0.6 ), Text Box( "   update? ->  draw a new line" ) ),
				Spacer Box( size( 100, 0 ) ),
				H List Box( Spacer Box( size( 0, 100 ) ), gb2_container = V List Box() )
			), 

		);
		LineScan:LinecutWindow << on Close( // doesn't work ?!??!
			Print( "closing window" );
			LineScan:CloseTemporaryTables();

		)
				
		;
	,
		LineScan:LineCutWindow << Bring Window To Front
	)
);

LineScan:LS_generateNewPlot = Function( {}, 
	//Print( "generatePlot" );
	Try( gbLC << delete() );
					
	gb2_container << Append(
		Eval(
			Substitute(
					Expr(
						gbLC = LineScan:dt_tempLineCut << Graph Builder(
							Transform Column( "YVariable", Formula( __statistics__ ) ),
							Size( 450, 250 ),
							Show Control Panel( 0 ),
							Show Legend( 0 ),
							Show Title( 0 ),
							Fit Window(),
							Graph Spacing( 4 ),
							__variables__,
							Elements( Points( X, Y, Legend( 3 ) ), Smoother( X, Y, Legend( 4 ), Lambda( 0.005 ) ) )
						)
					),
				Expr( __statistics__ ), Name Expr( LineScan:preparedYFunction ),
				Expr( __variables__ ), Name Expr( LineScan:preparedVariables )
			)
		)
	);
	If( N Items( LineScan:myGroups ),
		gbLC << Show Legend( 1 )
	);
	Report( gbLC )[AxisBox( 1 )] << Min( 0 );
	Report( gbLC )[Text Edit Box( 4 )] << Set Text( LineScan:YvariableName );
	Report( gbLC )[Outline Box( 1 )] << Set Title( "Linecut" );
	
);
	
LineScan:LS_update = Function( {},
	{Default Local}, 
	//Print( "LS: update" );
	If( LineScan:LS_findVariables(), 

		If( LineScan:LS_sel_rows(), 
				
			LineScan:LS_prepareExpressions( Linescan:myStatistics );
			LineScan:LS_prepareWindow();
			LineScan:LS_generateNewPlot();
		)
	)
);
	
LineScan:LS_getMousetrapCoordinates = Function( {},
	If( LineScan:MyfirstClick == 1, 
					
		LineScan:dt << Clear Select;
		LineScan:startx = x;
		LineScan:starty = y;
		LineScan:MyfirstClick++;
	);
	LineScan:exx = x;
	LineScan:exy = y;
);	


// now let's add the MouseTrap

LineScan:LS_Initialize();


Try(
	LineScan:CloseTemporaryTables();
	Linescan:TheReport = Current Report();
	gbrs = (Current Report() << XPath( "//OutlineBox[@helpKey = 'Graph Builder']" ));
	If( N Items( gbrs ),
		Linescan:gb = gbrs[1] << Get Scriptable Object();
		Linescan:dt = Linescan:gb << Get Data Table();	
		
		FrameBoxes = gbrs[1] << XPath( "//FrameBox" );
		For Each( {myFrameBox, idx}, FrameBoxes, 

			myFrameBox << add graphics script(
				Mousetrap(
					LineScan:LS_getMousetrapCoordinates(),
					LineScan:MyfirstClick = 1;
					LineScan:mouse_released = 1;
				);
	
				myline = Line( {LineScan:startx, LineScan:starty}, {LineScan:exx, LineScan:exy} );
	
				If( LineScan:mouse_released,
					If( 0,
						Print( 1 ),
						LineScan:LS_update()
					)
				);
				LineScan:mouse_released = 0;
			)
		);
	);
);

Re: Line Profile in JMP?

hogi — Fri, 18 Nov 2022 09:39:37 GMT

I just noticed that the selected data values (pink) for the linecut are slightly shifted with respect to the selection line (black).

At first sight, I thought that there is an error in the selection formula.

But it's the way the Heatmap plot is generated.

Let's say the x, y axes are defined with minor tick spacing dx / dy

Then the the Heatmap grid has the ame steps: dx, dy.

The color the tile at the position X -- X+dx / Y -- Y +dy is determined by aggregating all data points with X <= x < X+ dx, Y <= y<= Y + dy.

If there are many data points in this area, the approach is perfect.

But for unit steps (1 data point per tile; e.g. integer steps, 1, 2, 3, ...) one clearly see the offset. It comes from the asymmetry:

<= on the left.

< on the right.

neither of the symmetric versions

<= ... <=

< ... <

would be acceptable :)

Is there an option in JMP to shift the aggregation to

X -dx/2 <= x < X+dx/2,
Y -dy/2 <= y < Y+dy/2

From mathematical point of view, there won't be a benefit.

But for 99% of the use cases: (esp: integer , unit steps grids 1,2,3,4 ...) definitely yes ! :)

Re: Line Profile in JMP?

hogi — Tue, 27 Jun 2023 20:53:15 GMT

I added the Lincut function from this conversation to the GraphBuilderToolbar: