How to save Spearman Correlation Coefficients directly into JMP data table (scripting, JSL)

[#1]

Hi,
I've writen a small program that computes and saves Pearson coefficients directly into a JMP table.
Can you help adapt it to save Spearman coefficients in the same way? The "Spearman" option has no effect on storing ... one of th emany options I tried so far...

Thanks

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);


dt_2= report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_2 << save("C:\....\Spearman.jmp") ;

[#2]

Hi,

Does this do it?

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

[#3]

We'r getting close. I'm glad I learned about the "Outline Box" options.

Unfortunately, the output matrix shape is "triangular", not "square" (as needed; the "diagonal" 1.0s are missing because of this, too).

Is it easy to get there? 

[#4]

Hi agian,

It took a little manipulating, but see if you can make this work for you:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

variables={"Mass","Fore","Bicep","Chest"};
For(i=1,i<=4,i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);

Column(dt_spearman,1)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});
Column(dt_spearman,2)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});

dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];
		
	)
);

[#5]

Anytically speaking: Yes, it's the square matrix.You've done an impressive job, seriously.

From a technial point of views: This cannot work for me: I need these Spearman matrices for several datasets with 100s of fields.

I simply cannot change manually from triangular to square every time. I need to think of something else.

---

@HadleyMyers wrote:

Hi agian,

It took a little manipulating, but see if you can make this work for you:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

dt_2 = dt << Multivariate(
Y(
:Name( "Mass" ),
:Name( "Fore" ),
:Name( "Bicep" ),
:Name( "Chest" )
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;

variables={"Mass","Fore","Bicep","Chest"};
For(i=1,i<=4,i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=variables[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);

Column(dt_spearman,1)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});
Column(dt_spearman,2)<<Set Property("Value Ordering",{"Mass","Fore","Bicep","Chest"});

dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];
		
	)
);

---

[#6]

Would it be possible to generalize my script, or parts of it, so it can be used with your datasets?  If you can post one of your datasets after anonymizing the data (Table-->Anonymize), and create the multivariate script, maybe we can work something out?

[#7]

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

I have a better idea. Can you maybe generalize your approach to all columns in the "Body measurements" dataset (see link above)?

If we want to be bold, we could pretend the first column is an ID we want to skip... That would be the icing on the cake... 

[#8 · Accepted solution]

I think this should do it:

dt = Open("$SAMPLE_DATA/Body Measurements.jmp");

colnameslist=dt<<Get Column Names(string);

Remove From(colnameslist,1);


dt_2 = dt << Multivariate(
Y(
Eval(colnameslist)
),
Estimation Method( "Row-wise" ),
Matrix Format( "Square" ),
Scatterplot Matrix( 0 ),
Spearman's ρ(1)

);

report(dt_2)
[Outline Box("Correlations")][1, 1] << make into data table;

dt_spearman=Report( dt_2 )
[Outline Box( "Nonparametric: Spearman's ρ" )][Table Box( 1 )] <<
Make Into Data Table;


For(i=1,i<=Nitems(colnameslist),i++,
dt_spearman<<add rows(1);
Column(dt_spearman,1)[Nrows(dt_spearman)]=colnameslist[i];
Column(dt_spearman,2)[Nrows(dt_spearman)]=colnameslist[i];
Column(dt_spearman,3)[Nrows(dt_spearman)]=1;
);



dt_split= dt_spearman << Split(
	Split By( :Variable ),
	Split( :Spearman ρ ),
	Group( :by Variable ),
	Remaining Columns( Drop All ),
	Sort by Column Property
);

For(i=2,i<=Nrows(dt_split),i++,
	For(j=1,j<=Nrows(dt_split)-1,j++,
	
	if(is Missing(Column(dt_split,i)[j+1]),
	
	Column(dt_split,i)[j+1]=Column(dt_split,j+2)[i-1];,
	
	Column(dt_split,j+2)[i-1]=Column(dt_split,i)[j+1];
	)
	)
);

[#9]

Phantastic. 

Here some pointers for interested users:

JMP calculates correlations for retained fields only. Consequences of accidentally kept character columns:
(1) different structures: Spearman matrix keeps character columns; Pearson matrix filters them out automatically
(2) Missing values: Spearman does not calculate coefficients but places missing values

(3) Column order: Pearson: as in original dataset, Spearman: alphabetical.