[JSL] Get "Best Fit" Capability statistics from a data table and populate a new table

[#1]

Hello,

I have to manage big data files on a daily basis (10k columns x 20k rows).

My goal is to get capability statistics from all columns with limits, by using the best fit method (only a few columns are normally distributed).

I have made a quick and dirty script that:

- loop through columns with limits,

- plot distribution > "fits all" > capability,

- picks the info in the report window and

- close the report window

The code "seems" to work, but when I plot the metrics individually to verify if the new table matches the info, sometimes the results differ (not the same distribution type, not the same Ppk/Cpk values). On the other hand, USL, LSL and N are always right (so the script picks the info from the right window)

I don't know where the problem is coming from:

- I added some Wait() commands to make sure the processor can catch up before starting a new loop (but is it enough? is it the right way?)

- Is it because JMP Fits all function is not as precise as I expect?

- Is it something else?

I attached a script that generates a table with 15k rows and random formulas, followed by the actual code.

In this example, the first first should be SHASH but reports Johnson Sl in the table (with different capability). the 2x other are similar.

Thanks a lot for your help

Voiz

//Create table with random distribution 
dt = New Table( "tabletest",
	Add Rows( 15000 ),
	New Column( "Normal 10,1", Numeric, "Continuous", formula( Random Normal( 10, 1 ) ) ),
	New Column( "LogNormal 10,1", Numeric, "Continuous", formula( Random Lognormal( 10, 1 ) ) ),
	New Column( "Gamma 10,1", Numeric, "Continuous", formula( Random Gamma( 10, 1 ) ) ),
	New Column( "Normal 10,2", Numeric, "Continuous", formula( Random Normal( 10, 2 ) ) );
	
);
Column( dt, "Normal 10,1" ) << set property( "spec limits", {LSL( 6 ), USL( 14 ), Show Limits( 0 )} );
Column( dt, "LogNormal 10,1" ) << set property( "spec limits", {LSL( 0 ), USL( 400000 ), Show Limits( 0 )} );
Column( dt, "Gamma 10,1" ) << set property( "spec limits", {LSL( 0 ), USL( 40 ), Show Limits( 0 )} );

dt << save( "$Desktop\tabletest.jmp" );

//Get Column only with limits (At east LSL or USL, I don't care about target)
MetricCols = dt << Get Column Names( numeric, continuous );

Show( MetricCols );

For( i = N Items( MetricCols ), i >= 1, i--, 

	Q = Column( MetricCols[i] ) << Get Property( "Spec limits" );
	Show( Q );

	spec_x = Eval List( {Arg( Q, 1 )/*, Arg( Q, 2 ), Arg( Q, 3 )*/} );
	Show( spec_x );
	If( N Row( Loc( spec_x, Empty() ) ),
		Remove From( MetricCols, i )
	);
);
 
//Create a new table to collect the stats from the "Test" table
CpkTable = New Table( "Capability Best Fit",
	Add Rows( N Items( MetricCols ) ),
	New Column( "Metric", "Character", "Nominal" ),
	New Column( "N", "Numeric", "Continuous" ),
	New Column( "LSL", "Numeric", "Continuous" ),
	New Column( "USL", "Numeric", "Continuous" ),
	New Column( "Best Fit", "Character", "Nominal" ),
	New Column( "Cpk", "Numeric", "Continuous" ),
	New Column( "Cpl", "Numeric", "Continuous" ),
	New Column( "Cpu", "Numeric", "Continuous" )
);

//Loop to run Distribution > Fit ALL > Capability and collect the stats
For( i = 1, i <= N Items( MetricCols ), i++,
	Show( i );
	dt = Open( "$Desktop\tabletest.jmp" );
	dis = dt << Distribution(
		Continuous Distribution( Column( As Column( MetricCols[i] ) ), Fit Distribution( "All" ), Process Capability( 0 ), invisible )
	);
	Wait( 0 );
// Point to the first capability analysis and create a data table from it
	dtCpk = dis << report;
	Wait( 0 );
		// Fill USL and LSL in the new table
	Limits = Column( MetricCols[i] ) << Get Property( "Spec limits" );
	Try( CpkTable:LSL[i] = Limits["LSL"] );
	Try( CpkTable:USL[i] = Limits["USL"] );
		
		// Find N and fill the new table
	Try(
		FindN = dtCpk["Summary Statistics"][Number Col Box( 1 )] << get;
		CpkTable:N[i] = FindN[6];
	);
				
		// find the best fit and fill the new table
	Try(
		var = dtCpk["Compare Distributions"][String Col Box( "Distribution" )] << get;
		CpkTable:Best Fit[i] = var[1];
		CpkTable:Metric[i] = Char( MetricCols[i] );
	);

		// get CPK parameters from best fit capability analysis and fill the new table 
	Try(
		cpk = dtCpk["Fitted " || var[1]]["Quantile Sigma"][Number Col Box( "Index" )] << get;
		CpkTable:Cpk[i] = cpk[2];
		CpkTable:Cpl[i] = cpk[4];
		CpkTable:Cpu[i] = cpk[5];
		Show( MetricCols[i] );
		Show( var[1] );
		Show( cpk );
	);
	Wait( 0 );

	dtCpk << close window;
);

[#2 · Accepted solution]

I was able to replicate the issue.  I narrowed it down to an issue with "Fit Distribution( "All" )"  being specified within the Distribution command.  It appears that moving the "Fit Distribution" to a separate call fixes the discrepancy.

dis = dt << Distribution(
		Continuous Distribution( Column( As Column( MetricCols[i] ) )/*, Fit Distribution( "All" ), Process Capability( 0 )*/, invisible )
	);
dis << Fit All;

I suggest that you turn this issue into the Support Group at JMP

[#3]

Thanks Jim,

Indeed the solution works and now displays the right fit in the new table.

I will communicate with JMP Support group.

Is there a smart way in JSL to generate the Process Capability for the best fit from that point? (So I can populate the new table with the Cpk, Cpu...)

I could think of a dirty way to do it (find the fit, close the report, re-open report with the capability from the right fit) but if there is a smarter way to do it, I would prefer.

I tried a few variations of the code below, it doesn't work and I couldn't find anything in the scripting index, or in the discussions.

Thanks in advance,

Voiz

	dis = dt << Distribution(
		Continuous Distribution( Column( As Column( MetricCols[i] ) ),/*, Fit Distribution( "All" ),*/ Process Capability( 0 ), invisible )
	);
	dis << Fit All;
	dis << Process Capability ("Lognormal");