How to evaluate model accuracy on holdout set

[#1]

Hello,

I am using JMP 13 and I can not seem to find a way to automaticaly evaluate my model's accuracy on the holdout set. I am buliding a linear regression model.

So I would like to preserve the model (with the contained effects and estimates sizes), which was bulit on the training set, but calculate accuracy predictors (R^2, R^2 adjusted and RMSE) on the holdout set. In the moment the only solution I found is to calculate RMSE "by hand", since you can calculate the residuals. Is there any way to do this automaticaly?

This seems such a basic operation for model buliding  process so I am quite sure there should be some good option.

Thank you for your help,

Danijel

[#2]

The ability to automatically perform honest assessment of predictive models is a JMP Pro feature. The Fit Model launch dialog in JMP Pro has a Validation analysis role that is not available in JMP.

The Tabulate platform might make the estimation of the RMSE easier for you. Use your validation column for grouping and tabulate the standard deviation of the saved residuals.