cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
Danijel_V
Level III

How to evaluate model accuracy on holdout set

Hello,

 

I am using JMP 13 and I can not seem to find a way to automaticaly evaluate my model's accuracy on the holdout set. I am buliding a linear regression model.

So I would like to preserve the model (with the contained effects and estimates sizes), which was bulit on the training set, but calculate accuracy predictors (R^2, R^2 adjusted and RMSE) on the holdout set. In the moment the only solution I found is to calculate RMSE "by hand", since you can calculate the residuals. Is there any way to do this automaticaly?

 

This seems such a basic operation for model buliding  process so I am quite sure there should be some good option.

 

Thank you for your help,

Danijel

1 REPLY 1

Re: How to evaluate model accuracy on holdout set

The ability to automatically perform honest assessment of predictive models is a JMP Pro feature. The Fit Model launch dialog in JMP Pro has a Validation analysis role that is not available in JMP.

 

The Tabulate platform might make the estimation of the RMSE easier for you. Use your validation column for grouping and tabulate the standard deviation of the saved residuals.