cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
The Discovery Summit 2025 Call for Content is open! Submit an abstract today to present at our premier analytics conference.
See how to use to use Text Explorer to glean valuable information from text data at April 25 webinar.
Choose Language Hide Translation Bar
View Original Published Thread

k-fold r2

bbrown1
Level I

I am performing a stepwise procedure and evaluating the performance using a k-fold r-squared. Currently, the only k-fold r-square that is provided is averaged across all 5 folds. Is it possible to secure the r-squared for each individual k-fold in a model? 

1 REPLY 1
Victor_G
Super User


Re: k-fold r2

Hello @bbrown1,

In order to see the individual results for each fold, you can try to use instead the "Model Screening" platform ("Analyze", "Predictive Modeling", "Model Screening").
You'll enter your factors and response(s), check the "K-fold cross-validation", specify the number of folds, and simply check the "Fit Stepwise" in the menu "Methods".
Then, you should have your individual folds results for training and validation like in this screenshot : 

 

Victor_G_1-1651735401115.png

Be careful not to do "cherry picking" and choose to use the fold with the best results, as it will mean you are choosing the training and validation set in order to get the best results, which is not the target of this validation technique (and not the objective of a validation at all). It's more for assessing the robustness of a model in the context of small dataset, where no data is "lost" or excluded in the validation set.
You can find more information on cross-validation and model assessment in this discussion : Solved: How good is K-fold cross validation for small datasets? - JMP User Community 

I hope it will help you,

Victor GUILLER

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)

Recommended Articles

No recommendations found