K fold use options

[shampton82]

What inspired this wish list request? I don't use K fold validation as much as I feel I could due to the fact that it returns the best fitting fold, so I feel the model is showing better than I will see in "production".

What is the improvement you would like to see?

I would like to be able to have options on which fold to report such as best, worse, average/median, or even all (though this could be a dangerous option for large K folds)

Why is this idea important? K fold validation is a powerful tool available for both std and pro users but can be overly optimistic with the current options.

[SarahGilyard]

Thanks for posting shampton82. Can you provide a bit more detail about your request? Is there a specific type of model/modeling platform you want this to be applicable to? 

[shampton82]

Hey @SarahGilyard !

Yeah no problem.  

Here is an example for Neural, using Model screening I can see the fold but not in the actual platform.  

This would be a great help for JMP Standard users (in Neural) and offer more flexibility for Pro users who want to be able to use the expanded feature set in the native platform.  

I would like to see this for Neural, Bootstrap and Boosted forests, and SVM.

Hope this helps!

Steve

[SarahGilyard]

Thanks Steve for that clarification. Request is understood and I have captured it for some internal discussion.