Bring back the crossvalidation option in Partition platform menu

[VincentBechard]

Since JMP 16, either the normal and pro versions, the menu option for cross-validation is gone! To use cross-validation with decision trees, you need to use the Model Screening platform (new in JMP Pro only), and you can't modify any tree building option in there. This is more a downgrade than an upgrade! It was very convenient to have cross-validation along with all other options in a self-contained analysis platform.... Please bring it back!

[mia_stephens]

I believe you're referring to k-fold crossvalidation in Partition? You can turn this on in the preferences for Partition.

Correction: It was turned off as a default option in Partition because k-fold within the partition platform tends to result in overfitting - it doesn't stop soon enough. Within Model Screening k-fold is done external to the partition platform, so it works better there.

[VincentBechard]

Hi,

Yes I turned to option On. But there is a balloon tip saying "Deprecated feature", so I know it could disappear in the next releases. And, more important : it does not display the option in the menu of the platform, it's just a popup before the platform opens. In other words, it is not what I used to have before!

Is there a way or an addin that could bring back the menu?

Regards,

Vincent

[mia_stephens]

Hi Vincent,

Here's a little more information. The menu item was removed because k-fold in partition did not work as you might expect, and tended to result in models that did not generalize well to new data. As a result, these models aren't as useful. In studies, we found that random holdout in Partition (from the launch dialog) performed better on holdout data than k-fold. 

That said, thank you for submitting this wish for consideration. I've changed the status back to Open for Voting.

Warm regards,

Mia

[VincentBechard]

Hi,

I think "something" should be there for k-fold validation. Users without a Pro license lost an important feature! But you are true, the good old Holdout approach still works well, and works very fast on large datasets.

Thanks,

Vincent