Repeated k-fold Cross-Validation with Stepwise Regression (JSL Script?)

[#1]

Hi all,

Many of us know the variability that k-fold cross validated stepwise regression might have with small samples. It is often suggested to repeat this procedure 50-100 times in order to increase the realibility of the outputs.

I was wondering if there is a way in JMP to repeat this procedure? It will save me TONS of time!

THANK YOU

[#2 · Accepted solution]

Below is an example script that repeats k-fold crossvalidation on the same stepwise model 10 times. Copied from the JMP Scripting Index, the first part of this script runs a stepwise regression model and then performs a k-fold crossvalidation with 5 folds. The second piece makes the table box containing the step history and best model statistics into a data table for each iteration, and concatenates all the data tables together into one K-Fold Results table. You can of course choose to make a different table box of results into an output table, or none at all. 

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA/Fitness.jmp" );
For( i = 1, i <= 10, i++,
obj = dt << Fit Model( Y( :Oxy ), Effects( :Runtime, :Weight, :RunPulse, :RstPulse, :MaxPulse ), Personality( Stepwise ), Run );
obj << Name( "K-Fold Crossvalidation" )(5);
obj << Finish;

kfold = obj << report;
tablebox = kfold[Table Box( 3 )];

output = tablebox << Make Into Data Table( "K-Fold Results" );
If( i != 1,
Data Table( "K-Fold Results" ) << Concatenate( Data Table( "K-Fold Results " || Char( i ) ), Append to first table );
);
);