k-fold r2

bbrown1 — Fri, 09 Jun 2023 00:48:54 GMT

I am performing a stepwise procedure and evaluating the performance using a k-fold r-squared. Currently, the only k-fold r-square that is provided is averaged across all 5 folds. Is it possible to secure the r-squared for each individual k-fold in a model?

Re: k-fold r2

Victor_G — Thu, 05 May 2022 07:36:06 GMT

Hello @bbrown1,

In order to see the individual results for each fold, you can try to use instead the "Model Screening" platform ("Analyze", "Predictive Modeling", "Model Screening").
You'll enter your factors and response(s), check the "K-fold cross-validation", specify the number of folds, and simply check the "Fit Stepwise" in the menu "Methods".
Then, you should have your individual folds results for training and validation like in this screenshot :

Be careful not to do "cherry picking" and choose to use the fold with the best results, as it will mean you are choosing the training and validation set in order to get the best results, which is not the target of this validation technique (and not the objective of a validation at all). It's more for assessing the robustness of a model in the context of small dataset, where no data is "lost" or excluded in the validation set.
You can find more information on cross-validation and model assessment in this discussion : Solved: How good is K-fold cross validation for small datasets? - JMP User Community

I hope it will help you,

topic k-fold r2 in Discussions

k-fold r2

Re: k-fold r2