cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
Danijel_V
Danijel_V
Level III

How to evaluate model accuracy on holdout set

Sep 21, 2019 02:39 AM (2120 views)

Hello,

 

I am using JMP 13 and I can not seem to find a way to automaticaly evaluate my model's accuracy on the holdout set. I am buliding a linear regression model.

So I would like to preserve the model (with the contained effects and estimates sizes), which was bulit on the training set, but calculate accuracy predictors (R^2, R^2 adjusted and RMSE) on the holdout set. In the moment the only solution I found is to calculate RMSE "by hand", since you can calculate the residuals. Is there any way to do this automaticaly?

 

This seems such a basic operation for model buliding  process so I am quite sure there should be some good option.

 

Thank you for your help,

Danijel

0 Kudos
1 REPLY 1
Mark_Bailey
Staff Mark_Bailey
Staff

Re: How to evaluate model accuracy on holdout set

Sep 21, 2019 07:34 AM (2110 views)  |  Posted in reply to message from Danijel_V 09-21-2019

The ability to automatically perform honest assessment of predictive models is a JMP Pro feature. The Fit Model launch dialog in JMP Pro has a Validation analysis role that is not available in JMP.

 

The Tabulate platform might make the estimation of the RMSE easier for you. Use your validation column for grouping and tabulate the standard deviation of the saved residuals.

0 Kudos