topic Re: Col>modelling utilities> explore outlier scripting in Discussions

Col>modelling utilities> explore outlier scripting

ram_asra_gmail_ — Sun, 13 Mar 2016 05:29:06 GMT

HI All,

I have a question. Is it possible to have jsl script for" Col>modelling utilities> explore outlier" ?

Thanks

Ram

Re: Col>modelling utilities> explore outlier scripting

txnelson — Sun, 13 Mar 2016 13:14:28 GMT

The Explore Outliers window can be displayed with the following script:

Names Default To Here( 1 );

dt = open("$SAMPLE_DATA/Semiconductor Capability.jmp");

dt:npn1<<set selected(1);

Main Menu( "Explore Outliers" );

Re: Col>modelling utilities> explore outlier scripting

Wendy_Murphrey — Mon, 14 Mar 2016 18:25:07 GMT

Hello,

Current releases of JMP offer limited JSL support for Explore Outliers. See the options listed below. Further JSL support is planned for a future release of JMP.

dt = Open( "$SAMPLE_DATA/Probe.jmp" );
 
//Columns must be selected in order for any Explore Outliers options to launch
:VDP_PCOLL << Set Selected( 1 );
:VDP_PINNBASE << Set Selected( 1 );
:VDP_PINPBASE << Set Selected( 1 );
 
//Bring up specific Explore Outliers commands
obj = dt << Explore Outliers( Quantile Range Outliers( 1 ) );
obj = dt << Explore Outliers( Robust Fit Outliers( 1 ) );
obj = dt << Explore Outliers( Multivariate Robust Outliers( 1 ) );
obj = dt << Explore Outliers( Name( "Multivariate k-Nearest Neighbor Outliers" ) ); //Prompts interactively for k

Kind regards,

Wendy

Re: Col>modelling utilities> explore outlier scripting

txnelson — Mon, 14 Mar 2016 19:19:04 GMT

Wendy,

Where are these options documented? I don't see any reference in the Scripting Index, for either JMP 12 or JMP 13

Re: Col>modelling utilities> explore outlier scripting

Wendy_Murphrey — Mon, 14 Mar 2016 19:42:46 GMT

Hi, Jim.

For JMP 13, you should find the new syntax available in the Scripting Index under the Explore Outliers category. Documentation has not yet been completed for JMP 13. Previous versions did not have this functionality documented.

Wendy

Re: Col>modelling utilities> explore outlier scripting

txnelson — Mon, 14 Mar 2016 19:55:00 GMT

I looked and there it is, in JMP 13. Apparently I had only looked in JMP 12. Thank you

Re: Col>modelling utilities> explore outlier scripting

reshmimitra — Wed, 16 Mar 2016 21:34:58 GMT

Wendy,

I am using your example and able to get the Huber distance window up.

I am looking for outliers in column name = num_clk

dt << Distribution( ContinuousDistribution( Column( :num_clk ) ) ),

my_original_median = colquantile( :num_clk, .5 ),

my_original_mean = col mean ( :num_clk ),

:num_clk << Set Selected( 1 ),

obj = dt << Explore Outliers( Robust Fit Outliers ( ) ) << Huber K ( 1 ),

I have two questions after this:

How can I modify the K values here? It is going by default K=4 for my example. And, I want to explore this with multiple values of K.
How do I recalculate the mean and median of num_clk (column) after excluding the data points marked as outlier (by Huber above)? I have tried using the following, but I do not how many rows I will be excluding each time:

obj << Automatic Recalc( 1 );

dt << Select Rows( 5 ) << Exclude( 1 );

Any help will be highly appreciated.

Thanks,

Reshmi

Re: Col>modelling utilities> explore outlier scripting

Wendy_Murphrey — Thu, 17 Mar 2016 17:58:59 GMT

Hi, Reshmi.

In JMP 12, the only JSL methods specifically for Explore Outliers are what I provided earlier. After looking into your example, I found that I could navigate the window to change the value in the NumberEditBox and then click the Rescan button.

obj = dt << Explore Outliers( Robust Fit Outliers( 1 ) );

Report( obj )[Number Edit Box( 1 )] << Set( 3 );

Report( obj )[Button Box( 5 )] << Click;

The Col statistics functions are pre-evaluated statistics. In short, they are calculated once over the columns, without regard to the excluded rowstate, and are then used as constants so they do not need to be calculated again. This is explained in the "Pre-Evaluated Statistics" section of the Data Table chapter in the Scripting Guide.

The following formula will calculate the mean for only un-excluded rows:

Col Mean( If( Excluded(), ., :Viscosity ) );

Wendy

Re: Col>modelling utilities> explore outlier scripting

reshmimitra — Thu, 17 Mar 2016 22:26:27 GMT

Hi Wendy,

This was very helpful. Thank you!

I can now edit the value of K and see after the rescan.

My next action-item is to recalculate the mean and median after excluding the elements marked as outliers.

So, keeping that in mind, how can I script the following after the Rescan step:

1) Click the num_clk in Robust Estimates and Outliers (so that it highlights in the original table)

2) Click the "Exclude rows"

I am going to re_run this to get the new mean and median.

modified_median = colquantile( If( Excluded(), ., :Viscosity ), 0.5 );

modified_mean = Col Mean( If( Excluded(), ., :Viscosity ) );

Re: Col>modelling utilities> explore outlier scripting

Wendy_Murphrey — Fri, 18 Mar 2016 12:53:48 GMT

Hi, Reshmi.

How to click a button using JSL was demonstrated in my previous reply. To determine which ButtonBoxes to click, you will need to review the Display Tree to see the DisplayBoxes that make up the report. To access the Display Tree, right-click on the upper-most gray disclosure icon and select Edit > Show Tree Structure.

Good luck with your scripting project!

Wendy

Re: Col>modelling utilities> explore outlier scripting

ram_asra_gmail_ — Fri, 22 Apr 2016 02:30:08 GMT

Hello, Wendy. thanks for response.

Will be looking forward to see the full script functionality for this.

Rgds

Ram

Re: Col>modelling utilities> explore outlier scripting

johnm — Thu, 09 Feb 2017 16:44:41 GMT

Hello,

I very much like this outlier feature.

I understand there is limited scripting to this point in time.

In looking at the tree structure, I understand how to 'click' on a selection box.

But is there a way to select (click) a variable listed in the Column in the "OutlineBox(4): Robust Estimates and Outliers" ?

Thanks,
John

Re: Col>modelling utilities> explore outlier scripting

Wendy_Murphrey — Thu, 09 Feb 2017 18:16:18 GMT

Hi, John.

JMP 13 introduced more JSL support for Explore Outliers. See the Scripting Index (Help > Scripting Index) for details. However if you have not yet updated to JMP 13, you can still select rows by navigating to the TableBox. Below is an example to demonstrate:

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Water Treatment.jmp" );
dt << Select Column Group( "Sensor Measurements" );
obj = dt << Explore Outliers( Quantile Range Outliers( 1 ) );

Report( obj )[Table Box( 1 )] << Set Selected Rows( [1, 2] );

Hope that helps!

Re: Col>modelling utilities> explore outlier scripting

johnm — Thu, 09 Feb 2017 22:13:02 GMT

Hi Wendy,

Yes, this is exactly what I was wanting to know how to do.

Thanks very much,

John

Re: Col>modelling utilities> explore outlier scripting

ghartel — Tue, 11 Jul 2017 03:45:31 GMT

So, how owuld you do this in version 13? When I say reort(obj) << change to missing(); it says 'select a column...'. I could do the table box thing you showed, but I don't know how many columns will have missing values, or what their names would be. What the best way of scripting that in 13?

thanks

Re: Col>modelling utilities> explore outlier scripting

ghartel — Tue, 11 Jul 2017 04:35:12 GMT

I did the following but hoped something more elgant was available:

EO = Explore Outliers(
Y( Eval( VariableList ) ),
Show only columns with outliers( 1 ),
  Quantile Range Outliers
) << journal;
n = n items(Report( EO )[Table Box( 1 )][1] << get );
If( n,
  Report( EO )[Table Box( 1 )] << Set Selected Rows( 1 :: n );
  EO << Change to Missing();
);