topic Re: How to select column for Quantile Range Outlier using jsl in Discussions

How to select column for Quantile Range Outlier using jsl

XHLow — Sun, 11 Jun 2023 00:00:09 GMT

Hi JMP team,

I plan to do automated outlier analysis using Explore Outliers. The goal of this automation is to get a neat, easy to view table to show only row item and column item with outlier(This require to filter out those non-outlier column and row). Take Water Treatment as example below to rid row item without outlier.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Water Treatment.jmp" );
obj = dt << Explore Outliers(
	Y( dt << Get Column Group( "Sensor Measurements" ) )
);

obj << Show only columns with outliers(1);
obj << Tail Quantile( 0.1 );
obj << Q(3);
obj << Quantile Range Outliers;
obj << ColorCells(dt << Get Column Group( "Sensor Measurements" ));
obj << SelectRows(dt << Get Column Group( "Sensor Measurements" ));
obj << ExcludeRows(dt << Get Column Group( "Sensor Measurements" ));

dt << select where( Excluded(Row State()) != 1 );
dt << Delete rows;

Question how to filter out non-outlier column using jsl code?

Regards,

XH Low

Re: How to select column for Quantile Range Outlier using jsl

txnelson — Fri, 07 Apr 2023 03:56:33 GMT

I have added a few lines of code at the bottom of your script, that deletes the columns that have no outliers.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Water Treatment.jmp" );
obj = dt << Explore Outliers( Y( dt << Get Column Group( "Sensor Measurements" ) ) );

obj << Show only columns with outliers( 1 );
obj << Tail Quantile( 0.1 );
obj << Q( 3 );
obj << Quantile Range Outliers;
obj << ColorCells( dt << Get Column Group( "Sensor Measurements" ) );
obj << SelectRows( dt << Get Column Group( "Sensor Measurements" ) );
obj << ExcludeRows( dt << Get Column Group( "Sensor Measurements" ) );

dt << select where( Excluded( Row State() ) != 1 );
dt << Delete rows;

allEvaluatedList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;
obj << Show only columns with outliers( 1 );
colList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;

For Each( {col, i}, allEvaluatedList,
	If( Contains( colList, col ) == 0,
		dt << delete columns( col )
	)
);

Re: How to select column for Quantile Range Outlier using jsl

XHLow — Fri, 07 Apr 2023 07:28:40 GMT

Hi Jim,

Thanks for the code.

I tried the code above, my JMP15 did not support For Each(). I make some changes on For Each to For loop.

But still unable delete the non-outlier column.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Water Treatment.jmp" );
obj = dt << Explore Outliers( Y( dt << Get Column Group( "Sensor Measurements" ) ) );

obj << Show only columns with outliers( 1 );
obj << Tail Quantile( 0.1 );
obj << Q( 3 );
obj << Quantile Range Outliers;
obj << ColorCells( dt << Get Column Group( "Sensor Measurements" ) );
obj << SelectRows( dt << Get Column Group( "Sensor Measurements" ) );
obj << ExcludeRows( dt << Get Column Group( "Sensor Measurements" ) );

dt << select where( Excluded( Row State() ) != 1 );
dt << Delete rows;

allEvaluatedList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;
obj << Show only columns with outliers( 1 );
colList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;


For( i=1, i <= N Items(allEvaluatedList), i++, col=allEvaluatedList[i];
	If( Contains( colList, col ) == 0 ,
		dt << delete columns( col )
	)
);

Non outlier column still present in the data table below.

Am I missing something?

Re: How to select column for Quantile Range Outlier using jsl

txnelson — Fri, 07 Apr 2023 10:22:33 GMT

The output of the Explore Outliers Platform in JMP 15, by default, only displays the columns that have outliers. The script I provided assumed that all columns that were specified for analysis were displayed. I have modified the code to force the displaying of all columns so that the "allEvaluatedList" gets properly populated. I tested the code in JMP 15, and iit is now working.

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Water Treatment.jmp" );
obj = dt << Explore Outliers( Y( dt << Get Column Group( "Sensor Measurements" ) ) );

obj << Show only columns with outliers( 1 );
obj << Tail Quantile( 0.1 );
obj << Q( 3 );
obj << Quantile Range Outliers;
obj << ColorCells( dt << Get Column Group( "Sensor Measurements" ) );
obj << SelectRows( dt << Get Column Group( "Sensor Measurements" ) );
obj << ExcludeRows( dt << Get Column Group( "Sensor Measurements" ) );

dt << select where( Excluded( Row State() ) != 1 );
dt << Delete rows;

obj << Show only columns with outliers( 0 );
allEvaluatedList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;
obj << Show only columns with outliers( 1 );
colList = Report( obj )["Quantile Range Outliers", String Col Box( "Column" )] << get;


For( i=1, i <= N Items(allEvaluatedList), i++, col=allEvaluatedList[i];
	If( Contains( colList, col ) == 0 ,
		dt << delete columns( col )
	)
);

Re: How to select column for Quantile Range Outlier using jsl

XHLow — Fri, 07 Apr 2023 12:30:07 GMT

Great!! It works now.

Thanks Jim.