Share your ideas for the JMP Scripting Unsession at Discovery Summit by September 17th. We hope to see you there!
Choose Language Hide Translation Bar
Highlighted
Danijel_V
Level III

How to evaluate model accuracy on holdout set

Hello,

 

I am using JMP 13 and I can not seem to find a way to automaticaly evaluate my model's accuracy on the holdout set. I am buliding a linear regression model.

So I would like to preserve the model (with the contained effects and estimates sizes), which was bulit on the training set, but calculate accuracy predictors (R^2, R^2 adjusted and RMSE) on the holdout set. In the moment the only solution I found is to calculate RMSE "by hand", since you can calculate the residuals. Is there any way to do this automaticaly?

 

This seems such a basic operation for model buliding  process so I am quite sure there should be some good option.

 

Thank you for your help,

Danijel

1 REPLY 1
Highlighted

Re: How to evaluate model accuracy on holdout set

The ability to automatically perform honest assessment of predictive models is a JMP Pro feature. The Fit Model launch dialog in JMP Pro has a Validation analysis role that is not available in JMP.

 

The Tabulate platform might make the estimation of the RMSE easier for you. Use your validation column for grouping and tabulate the standard deviation of the saved residuals.

Learn it once, use it forever!
Article Labels

    There are no labels assigned to this post.