topic Transform Column: Bug with excluded rows in Discussions

Transform Column: Bug with excluded rows

hogi — Fri, 27 Oct 2023 11:03:36 GMT

In the below example, I use a Data Filter to restrict the data - and a Local Data Filter inside the GraphBuilder Script is used to further restrict the data.

3 different Ranks are plotted:

The fist Rank (blue points) doesn't have an Excluded() argument and therefore all data points are counted - even ones which are excluded by the (global) Data Filter. This is why the first plotted entry already starts at a high value.
The Second Rank formula (red) has an Excluded() argument, therefore some rows don't count and the Rank values are lower. But still, they don't start at 1. This is expected, because the Formula is stored in the Data Table - and doesn't know about the rows which are excluded by the Local Data Filter of the Plot.
The third Rank formula is the same as the second one, but it is used in a Transform Column which is generated with the report. Unfortunately, also this formula ignores the rows that are excluded by the Local Data Filter.

not shown:
the rank as the user expects it: values from 1 to N.

Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

dt << Data Filter( Add Filter( columns( :height ), Where( :height >= 63 ) ) );


dt << Graph Builder(
	Transform Column( "Rank[height] 3", Formula( Col Rank( :height, Excluded() ) ) ),
	Variables(
		X(
			:name,
			Order By( :"Rank[height]"n, Ascending, Order Statistic( "Mean" ) )
		),
		Y( :"Rank[height]"n ),
		Y( :"Rank[height] 2"n, Position( 1 ) ),
		Y( :"Rank[height] 3"n, Position( 1 ) )
	),
	Elements( Points( X, Y( 1 ), Y( 2 ), Y( 3 ), Legend( 14 ) ) ),
	Local Data Filter( Add Filter( columns( :height ), Where( :height >= 65.815 ) ) ),

)

Re: Transform Column: Bug with excluded rows

hogi — Sun, 10 Sep 2023 09:01:48 GMT

As a workaround, one could ask the Data Filter for the filtered rows and adjust the Formula Column accordingly.

It works - but definitely not with the comfort one is used to when working with GraphBuilder and JmpDataFilter.

A big disadvantage:

In Jmp, users have the freedom to add a Local Data Filter at any time.

Who will add the filter state handler to guarantee that the displayed values are updated?

A combination of a Formula Column AND a Data Filter - this doesn't sound like a niche application.

So, besides this workaround, I guess there is an easy solution available in GraphBuilder - but I couldn't find it up to now.

Who had a similar application - and found already the 1-click Jmp comfort solution?

Names Default to Here(1);
dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

myRows=1::N Rows(dt);

New Column( "Rank_correct",Formula( Col Rank( :height, Excluded(), Contains( myRows, Row() ) > 0 ) ));

gb = dt << Graph Builder(
Variables(
		X(
			:name,
			Order By( :Rank_correct, Ascending, Order Statistic( "Mean" ) )
		),
		Y( :Rank_correct )
	),Elements( Points( X, Y ) ));
	
updateNow = Function({x},
		myRows= (Current Report()["Local Data Filter"] << get scriptable object) << Get Filtered Rows;
        if(or(isempty(myRows),NItems(myRows)==0),myRows=1::N Rows(dt));
		:Rank_correct<<Eval Formula
);
	
ldf = gb << Local Data Filter(	Add Filter(	columns( :name ),	Where( :name == {"AMY", "CLAY", "EDWARD"} )	));
	

updateNow(1);
rs = (Current Report()["Local Data Filter"] << get scriptable object) << Make Filter Change Handler( updateNow );

Re: Transform Column: Bug with excluded rows

hogi — Tue, 12 Sep 2023 11:31:22 GMT

I posted a wish to enhance the functionality of Transform Columns in Graph Builder (and other reports like Summary !!!):

With the new functionality Transfrom Columns will get as comfortable as Summary Statistics - and the functionality will go far beyond that :)