topic Re: How do I make one Data Filter mirror another? in Discussions

How do I make one Data Filter mirror another?

TomF — Thu, 31 Oct 2024 16:06:47 GMT

I have a window containing two graph builders, pulling from two different datasets, and I want to filter both graphs with a single Data Filter. I don't think this is possible. I've thought about instead using two "synced" data filters, but couldn't figure it out. Last week at JMP Discovery, @danschikore pointed me toward event handlers as a possible solution for using two synced filters. I think I'm close to getting it to work, but I'm stuck.

The script below creates the two filters and two graphs. I want the selections made in the top filter to be automatically reflected in the bottom filter. Once that works, I will hide the bottom filter and the user will see one filter controlling both graphs.

As it is now, when the user makes a selection in the top filter, the script prints that Where clause to the log using "Get Where Clause", but I need to apply this where clause to the bottom filter. I want a command like "Set Where Clause" that would let me shove that where clause text into the bottom filter, but it doesn't seem to exist. I'm really hoping to avoid the scenario where I have to "unpack" the text string (below) and turn it into a form I can pass to the lower filter.
Select Where( :name == \!"ALICE\!" | :name == \!"BARBARA\!" )

Just to convince myself I can get the first filter to control the second one in some fashion, I have the first filter clearing the second filter's selection. That means I can talk to both filters; I can extract the where clause from the first; I just can't apply it to the second.

As a bonus question: Once this syncing works. I want to hide the second filter. The last line of the script hides the first filter. Does anyone know how I could modify this to hide the second filter instead?

Thanks,

Tom

Re: How do I make one Data Filter mirror another?

hogi — Thu, 31 Oct 2024 20:30:04 GMT

Nice : )
When I created my first State Handler, I forgot to store the filter state handler in a variable - and it got destroyed immediately.
And then it didn't work because I used a function with 0 arguments. -> well done, you escaped both trapdoors : )

There is no Set where clause - the Where Clause is no message which a Data Filter will understand, you can send it to the table to select the rows. Besides that << Get Where Clause () returns a string. You have to use Parse to convert it to an expression.

Unpack is less effort than you might expect - just ask the data filter for the script, extract the Add Filter() part via Extract Expr() and apply it to the second filter - will work for simple selections *)

eh = df1_Wt << MakeFilterChangeHandler(
	Function( {x},
		dfscript = df1_Wt << get script(); 	// store the code to create DF1 - with all the settings
		filter = Extract Expr(
			dfscript,    					// take the code which is stored in dfscript and ...
			Add Filter( Wild List() )		// search for the part with Add Filter(something). Wild List() - like magic :)
		);         

		Eval(                  				// run the code
			Eval Expr(             			// search for any Expr() and execute it (*)
				df2_Ht << delete all() 		// remove all previous settings
				<< Expr(             		// (*) find this Expr() and execute it
					Name Expr( filter )     // take the code which is stored in "filter" and replace filter with the actual code
				)
			)
		);
	)

);

*) When you combine several filters via OR(), the script will contain multiple add Filter() - and Extract Expr() will just return the first one.
To get an ultimate clone function, you can parse the DF script, collect all AddFilter() and apply them one after the other to the second Data Filter.

Re: How do I make one Data Filter mirror another?

jthi — Thu, 31 Oct 2024 16:43:50 GMT

You can filter two (or more) graphs with single filter by using Data Filter Context/Source box BUT they have to have same source table

Names Default To Here(1);
dt = Open("$SAMPLE_DATA/Big Class.jmp");
nw = New Window("Selection Filter",
	Data Filter Context Box(
		H List Box(
			dfsb = Data Filter Source Box(
				dt << Data Filter(
					Local,
					Add Filter(columns(:age), Display(:age, N Items(6)))
				),
			),
			Graph Builder(
				Size(208, 207),
				Show Control Panel(0),
				Show Legend(0),
				Variables(X(:age)),
				Elements(Bar(X, Legend(3)))
			),
			Bubble Plot(
				X(:weight),
				Y(:height),
				Sizes(:age),
				Title Position(0, 0)
			)
		)
	)
);

Now my questions is what you are trying to do? You wish to have one filter which filters two reports FROM different tables? I would use one local data filter and then use that to hide/exclude rows from the second table (if you allow it to modify the table instead of being just local). If it has to be local there are still other options than going with two local data filters (which is also possible).

Re: How do I make one Data Filter mirror another?

hogi — Thu, 31 Oct 2024 17:13:18 GMT

Make sure that both tables contain the same values!
If the data filter doesn't find a matching row, it will reset and show all rows:

This issue will be fixed in JMP19 via

As a workaround for <=JMP18, you can add dummy rows with missing data in the remaining columns
-> won't show up in the plot but will prevent the data filter from getting reset.

//Data Filter Event Handler Test.jsl
//2024-10-31

dtHeight = Open( "$SAMPLE_DATA/Big Class.jmp" );
dtHeight << delete columns({"age", "sex", "weight"});
dtHeight << Set Name ("Height");

dtWeight = Open( "$SAMPLE_DATA/Big Class.jmp" );
dtWeight << delete columns({"age", "sex", "height"});
dtWeight << Set Name ("Weight");


dtHeight << select where(:Name == "ALFRED") << delete  rows(); // <- dangerous !!!!

Try(win << Close Window);
win = New Window("Height and Weight",
	Data Filter Context Box(
		H List Box (
			V List Box(
				df1_Wt = dtWeight << Data Filter( Local,
					Show Controls( 0 ),	Show Modes( 0 ), Mode( Include( 1 ) ),
					Add Filter(Columns(:name)),
					Display( :name, N Items( 4 ))
				),
				df2_Ht = dtHeight << Data Filter( Local, 
					Show Controls( 0 ),	Show Modes( 0 ), Mode( Include( 1 ) ),
					Add Filter(Columns(:name)),
					Display( :name, N Items( 4 ))
				)
			),
			V List Box (
				gbWt = dtWeight << Graph Builder(
					Size( 612, 250 ),
					Show Control Panel( 0 ), Show Legend (0), Show Title (0), 
					Show X Axis (0), Show X Axis Title (0), Show Footer( 0 ),
					Variables( X( :name ), Y( :weight ) ),
					Elements( Bar( X, Y, Legend( 6 ) ) ),
					//Dispatch( {}, "Graph Builder", OutlineBox, {Set Title( "" )} ),
					Dispatch({},"weight",ScaleBox,{ Max( 200 )})
				),
				H List Box(
					Spacer Box(Size(25,200)),				
					gbHt = dtHeight << Graph Builder(
						Size( 600, 300 ),
						Show Control Panel( 0 ), Show Legend (0), Show Title (0),
						Variables( X( :name ), Y( :Height ) ),
						Elements( Bar( X, Y, Legend( 6 ) ) ),
						Dispatch( {}, "Graph Builder", OutlineBox, {Set Title( "" )} )
					)
				)
			)
		)
	)
);

eh = df1_Wt << MakeFilterChangeHandler(
	function({x},
dfscript = df1_Wt << get script();
filter = Extract Expr(dfscript, Add Filter(Wild List()));
Eval(Eval Expr(df2_Ht << delete all() << Expr (Name Expr(filter)))))

);

Re: How do I make one Data Filter mirror another?

hogi — Thu, 31 Oct 2024 17:25:57 GMT

@jthi wrote:
If it has to be local there are still other options than going with two local data filters (which is also possible).

The filter state handler is not just triggered by changes of the data filter.
Your bar graphs are save - but for point plots, the State Handler will be triggered whenever the mouse touches a data point!

So, depending on the plot type, it might be a good idea to compare the current filter settings with the previous ones - and just apply the settings to the second plot when you detect a change.

maybe vote here:

Re: How do I make one Data Filter mirror another?

TomF — Thu, 31 Oct 2024 19:55:32 GMT

@hogi I don't completely understand what your code does, but I put it in and it worked perfectly. Thank you!

To be honest, I did not really escape the trap door. I fell in and then pulled myself back out. I was following this community post, but I saw no reason to store the event handler as a variable, so I didn't. It didn't work, I followed the post more carefully, storing it as a variable, it started working, and I carried on without understanding what happened. I put comments in my code with your two trapdoors, hoping to save someone the bruises in the future.

In this case, my filtering is simple -- always one column -- and both my tables contain the same x values, so no concerns there.

I've posted the updated code above.

Re: How do I make one Data Filter mirror another?

jthi — Thu, 31 Oct 2024 19:27:16 GMT

There are plenty of different options for something like this and all depends on what is really being done (how the data looks like, can row states be manipulated, what is being filtered (and why), ...). Here is one option which works (usually) quite well in many case but it has quite complicated function for the handler (adding fixed size (with the help of Spacer Box()) for something like this is generally a good idea to avoid annoying blinking of the window).

Names Default To Here(1);

dt1 = Open("$SAMPLE_DATA/Probe.jmp");
dt2 = dt1 << Subset(All Rows, Selected Columns(0));

dt2 << Delete Rows(dt2 << get rows where(:Process == "Old"));

nw = New Window("",
	Data Filter Context Box(
		H List Box(
			Data Filter Source Box(
				df1 = dt1 << Data Filter(
					Local, 
					Show Controls(0), 
					Show Modes(0), 
					Mode(Include(1)), 
					Add Filter(Columns(:Site, :Process)), Display(:Site, N Items(5))
				),
			),
			V List Box(
				gb1 = dt1 << Graph Builder(
					Size(525, 458),
					Show Control Panel(0),
					Variables(X(:Lot ID), Y(:DELL_RPNBR)),
					Elements(Bar(X, Y, Legend(4)))
				),
				gb2_collector = V List Box()
			)
		)
	)
);


create_gb2 = function({a},
	c = df1 << Get where clause;
	If(Is Missing(c), // could also just use dt2 instead of dt2_private
		dt2_privatenew = dt2 << Subset(All Rows, Selected columns only(0), Link to original data table(1), private);
	,
		where = Arg(Parse(df1 << Get where clause));
		dt2_privatenew = Eval(EvalExpr(dt2 << Subset(
			Filtered Rows(Expr(Name Expr(where))),
			Selected columns only(0),
			Link to original data table(1),
			private
		)));
		If(Is Empty(dt2_privatenew), // no rows left -> pick first row and remove it, no linking
			dt2_privatenew = dt2 << Subset(Rows(1), Selected columns only(0), Link to original data table(0), private);
			dt2_privatenew << Delete Rows(1);
		);
	);
	
	nw << inval;
	Try(Close(dt2_private, no save)); // will also remove old graph builder
	dt2_private = dt2_privatenew;
	
	gb2_collector << Append(
		gb2 = dt2_private << Graph Builder(
			Size(528, 458),
			Show Control Panel(0),
			Variables(X(:Lot ID), Y(:DELL_RPNBR)),
			Elements(Bar(X, Y, Legend(20)))
		)
	);
	wait(0);
	nw << Update Window;
);

create_gb2(.);

nw:handler = df1 << Make Filter Change Handler(
	create_gb2();
);

Eval(EvalExpr(
	nw << On Close(
		Try(Close(Expr(dt1), no save));
		Try(Close(Expr(dt2), no save));
		Try(Close(Expr(dt2_private), no save));
	);
));

Write();

Re: How do I make one Data Filter mirror another?

TomF — Thu, 31 Oct 2024 19:41:35 GMT

Yes, I want one filter that filters two reports from different tables. I imagine that doesn't make sense with the simple Big Class example I shared above, but I'm attaching a more complicated example that closely mimics the structure of my actual data set.

Imagine you have one table with the current height of each student and another table with the weight of each student for the past three years. This is how I want to view that data:

The height data set obviously has one row per student. The weight data starts with one row per student (three columns of weight), but there is no way I know of in Graph Builder to make that bottom graph without stacking the weights. My weight table now has 3X as many rows as the height table.

Because a data filter can't filter two different tables, my solution so far was to concatenate the tables, just to be able to make this graph. That isn't elegant, but it works fine.

Where I really got in to trouble was that when I hover over the height I want the pop-up to show if they were wearing shoes during the measurement, and when I hover over the weight I want to know the serial number of the scale that was used. When all the data is in one table, and I make Label(1) for both columns you end up with missing values when you hover (because there's no scale serial number on a weight measurement)...which can also be addressed, but it just started to get very messy.

I think the solution of leaving the data in two tables with two synced filters (one hidden) is going to work much better. But if there's a better way to create this graph I'm open to suggestions.
Just a general shout out to Graph Builder and the ability to control it with JSL. This seamless melding of two different graphs is only possible because we have the ability to turn off everything on the bottom of the top graph and top of the bottom graph and bring them right up against each other. This has allowed us to create a very customized and very data-dense graphic in our specific situation. And now we can filter it elegantly too:) Maybe others will find this useful down the road.

Re: How do I make one Data Filter mirror another?

TomF — Thu, 31 Oct 2024 20:05:43 GMT

I'm afraid a lot of that code is beyond me, but I get the general idea. I will keep this in mind for more complicated situations. Thanks!

Re: How do I make one Data Filter mirror another?

hogi — Thu, 31 Oct 2024 20:27:54 GMT

Hi @TomF , I added some comments to the code to explain the individual steps.

One trick is the Extract Expr() - it scans the expression and searches for a specific pattern.

I just had a look at an old script where I used one data filter to control 4 additional data filters in the same Dashboard.
And it looked like

		For( i = 1, i <= N Arg( filterScript ), i++,
			If( Head( Arg( filterScript, i ) ) == Expr( Add Filter() ),
				filterExpression = Arg( filterScript, i )
			)
		);

The approach via Extract Expr() is much more elegant - I just learned it last year from @jthi in this discussion:

The other trick: Eval (Eval Expr( Expr())) - a Swiss army knife!
It will solve 95% of your issues in JSL :)

Re: How do I make one Data Filter mirror another?

jthi — Thu, 31 Oct 2024 20:28:36 GMT

Knowing different filtering techniques make it possible to do all sorts of nice things in JMP (sometimes they are also necessary especially if you have lots of data).

I assume that by concatenate you might mean update/join and we could have single table like this (virtual join can be used for this if there is unique identifier)

From this it is possible to create a graph like this

You can do it for example by utilizing Textlets (hover label)

Graph Builder(
	Size(1085, 688),
	Show Control Panel(0),
	Show Legend(0),
	Variables(X(:Name), X(:Year, Position(1)), Y(:Height), Y(:Weight), Overlay(:Name)),
	Elements(Position(1, 1), Bar(X(1), X(2), Y, Overlay(0), Legend(13))),
	Elements(Position(1, 2), Line(X(1), X(2), Y, Legend(15)), Points(X(1), X(2), Y, Legend(16))),
	Local Data Filter(
		Add Filter(
			columns(:Name),
			Where(
				:Name == {"AMY", "BARBARA", "CAROL", "CHRIS", "CLAY", "DANNY", "DAVID", "EDWARD", "ELIZABETH", "FREDERICK", "HENRY", "JACLYN"}
			),
			Display(:Name, N Items(15), Find(Set Text("")))
		)
	),
	SendToReport(
		Dispatch({}, "Name", ScaleBox, {Label Row(2, Show Major Grid(1))}),
		Dispatch({}, "Graph Builder", FrameBox, {Set Textlet(Markup("Serial Number: {:Serial No.[local:_firstRow]}"))}),
		Dispatch({}, "Graph Builder", FrameBox(2), {Set Textlet(Markup("Serial Number: {:Serial No.[local:_firstRow]}"))})
	)
);

You can customize the text shown much more if necessary, Using JMP > JMP Reports > Customize Hover Labels in JMP Graphs > Add Text to Hover Labels in JMP Graphs

Re: How do I make one Data Filter mirror another?

hogi — Thu, 31 Oct 2024 20:49:08 GMT

Thanks @TomF - very cool trick to fight against the restriction of label settings: use 2 tables : )
I wonder why it's not possible to generate plots like this - neither in the separate graphs, nor combined in one graph.

setting the label for one graph automatically forces the other graphs to show the labels as well : (

Re: How do I make one Data Filter mirror another?

TomF — Mon, 04 Nov 2024 19:48:50 GMT

I did mean concatenate. By doing update/join you are associating the height data, which has no year associated with it, with the weight data, which is yearly. The new graph makes it look like Barbara's height was measured to be 60 inches in 2022, 23, and 24, which it was not.
The trick here is that the data in the bottom is associated with year, but the data in the top is not, and I want one centered bar for it. Maybe instead of height I should have used "GDP per capita when and where the student was born" -- clearly not associated with 22/23/24.
Because the underlying data for the top and bottom graphs have a different structure, I think I need two separate graph builders to create the picture I want (above).

Re: How do I make one Data Filter mirror another?

TomF — Mon, 04 Nov 2024 19:56:41 GMT

@hogi, thanks for these links. Eval/Eval Expr() has been the most frustrating part of JSL for me over the past few years. I've used it successfully a few times by copying and tweaking examples from the Community, but never felt like I actually grasped it well. I look forward to going through the list of resources you've collected and maybe that will help it finally sink in. I know it's something I need to add to my toolkit.

Re: How do I make one Data Filter mirror another?

hogi — Mon, 04 Nov 2024 20:01:29 GMT

Agree : )

For Big Class there might be some easy solution, but this solution might not help with the actual issue.

Re: How do I make one Data Filter mirror another?

hogi — Mon, 04 Nov 2024 20:11:47 GMT

Eval(): run the code *)

Eval Expr: pre-evaluate everything wrapped with Expr() *)

Expr(): marker for code which has to get pre-evaluated *)

*) my brain : "the E word" - 3 times.

and the 4th one is Expr Name() ; -)