topic Re: Select middle 90% range using quantiles in Discussions

Select middle 90% range using quantiles

rverma — Fri, 09 Jun 2023 22:05:14 GMT

I have a Numeric Continuous column labeled as Height with several hundred height values. I want to exclude outliers by just using the middle 90% range of data. I am trying to script it in JSL by selecting all rows below 5% quantile and above 95% quantile and then exclude them. But I am not able to make it work by using script below. I will appreciate any guidance.

RowsBelow5PCTL = dt << get rows where( :HEIGHT < Col Quantile( :HEIGHT, .05 ) );

RowsAbove95PCTL = dt << get rows where( :HEIGHT > Col Quantile( :HEIGHT, .95 ) );

Re: Select middle 90% range using quantiles

txnelson — Fri, 05 Feb 2021 04:44:42 GMT

I believe this will give you what you want;

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/big class.jmp" );

dt << select where(
	:HEIGHT < Col Quantile( :Height, .05 ) | 
	:HEIGHT > Col Quantile( :Height, .95 )
);
dt << exclude;

Or if you need to have your above and below row number matrices, then this will work

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/big class.jmp" );

RowsBelow5PCTL = dt << get rows where( :HEIGHT < Col Quantile( :HEIGHT, .05 ) );

dt << select rows(RowsBelow5PCTL);

RowsAbove95PCTL = dt << get rows where( :HEIGHT > Col Quantile( :HEIGHT, .95 ) );
dt << select rows(RowsAbove95PCTL);

dt << exclude;

Re: Select middle 90% range using quantiles

rverma — Fri, 05 Feb 2021 19:09:20 GMT

Thank you very much.