topic Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL) in Discussions

How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

Newbie2Jumpie — Wed, 13 Nov 2019 08:28:19 GMT

Hi,
I've writen a small program that computes and saves Pearson coefficients directly into a JMP table.
Can you help adapt it to save Spearman coefficients in the same way? The "Spearman" option has no effect on storing ... one of th emany options I tried so far...

Thanks

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);


dt_2= report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_2 << save("C:\....\Spearman.jmp") ;

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

HadleyMyers — Wed, 13 Nov 2019 08:47:21 GMT

Hi,

Does this do it?

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

Newbie2Jumpie — Wed, 13 Nov 2019 10:29:42 GMT

We'r getting close. I'm glad I learned about the "Outline Box" options.

Unfortunately, the output matrix shape is "triangular", not "square" (as needed; the "diagonal" 1.0s are missing because of this, too).

Is it easy to get there?

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

HadleyMyers — Wed, 13 Nov 2019 11:39:35 GMT

Hi agian,

It took a little manipulating, but see if you can make this work for you:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

variables={"Mass","Fore","Bicep","Chest"};
For(i=1,i<=4,i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);

Column(dt_spearman,1)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});
Column(dt_spearman,2)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});

dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];
		
	)
);

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

Newbie2Jumpie — Wed, 13 Nov 2019 11:59:50 GMT

Anytically speaking: Yes, it's the square matrix.You've done an impressive job, seriously.

From a technial point of views: This cannot work for me: I need these Spearman matrices for several datasets with 100s of fields.

I simply cannot change manually from triangular to square every time. I need to think of something else.

@HadleyMyers wrote:

Hi agian,

It took a little manipulating, but see if you can make this work for you:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

variables={"Mass","Fore","Bicep","Chest"};
For(i=1,i<=4,i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);

Column(dt_spearman,1)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});
Column(dt_spearman,2)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});

dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];
		
	)
);

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

HadleyMyers — Wed, 13 Nov 2019 12:17:33 GMT

Would it be possible to generalize my script, or parts of it, so it can be used with your datasets? If you can post one of your datasets after anonymizing the data (Table-->Anonymize), and create the multivariate script, maybe we can work something out?

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

Newbie2Jumpie — Wed, 13 Nov 2019 12:43:55 GMT

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

I have a better idea. Can you maybe generalize your approach to all columns in the "Body measurements" dataset (see link above)?

If we want to be bold, we could pretend the first column is an ID we want to skip... That would be the icing on the cake...

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

HadleyMyers — Wed, 13 Nov 2019 13:11:15 GMT

I think this should do it:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

colnameslist=dt<<Get Column Names(string);

Remove From(colnameslist,1);


dt_2 = dt << Multivariate(
Y(
Eval(colnameslist)
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;


For(i=1,i<=Nitems(colnameslist),i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=colnameslist[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=colnameslist[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);



dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	if(is Missing(Column(dt_split,i)[j+1]),
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];,
	
	Column(dt_split,j+2)[i-1]=Column(dt_split,i)[j+1];
	)
	)
);

Re: How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

Newbie2Jumpie — Thu, 21 Nov 2019 09:43:10 GMT

Phantastic.

Here some pointers for interested users:

JMP calculates correlations for retained fields only. Consequences of accidentally kept character columns:
(1) different structures: Spearman matrix keeps character columns; Pearson matrix filters them out automatically
(2) Missing values: Spearman does not calculate coefficients but places missing values

(3) Column order: Pearson: as in original dataset, Spearman: alphabetical.