How to use a validation column in normal JMP15 (not JMP Pro)

[#1]

Hi,

I would like to build predictive models using Partition, Logistic Regression and Neural Networks for my categorical response Y (Pass or Fail) in JMP 15. I do not have the JMP Pro version.

I was able to create a validation column containing Training, Validation and Test sets.

How can I use the validation column in the normal version of JMP when building and evaluating the models?

Partition and Neural Network platform offer built-in "Validation" settings. However it will not use my validation column where I also have test rows.

The Fit Model platform does not offer any settings for validation in the normal version of JMP.

Any help on how to use my validation column in these platform using the normal version of JMP would be greatly appreciated.

Thanks!

[#2]

Did you have a look at:

Validation in JMP and JMP Pro

You can do it manually, excluding the rows for validation, and after the model has been built deploy it and compare the results (save the predicted formula, and let it calculate predictions also for the validation set).

[#3]

Thanks for the information but something still is not clear to me.

I build my model with the unexcluded rows which are the training set and save the prediction formula. I calculate the prediction for the validation set and compare it to the training set.

What about the test set? What am I going to do with the test set now and how does it differ from the validation set in this case?

[#4 · Accepted solution]

Yes, I understand that problem, the terms are a little confusing.

Usually the Training set is used to build the model, and the validation set is also part of the model building process (to tune the model, to choose the right model, to avoid overfitting).

So both the training and validation set are used to define the model.

And the test set should be fully new to the model. It is to judge the performance of the model on never seen data.

So in your case you can understand the validation set as the test set, you only need two different sets, one for building the model, the other for testing it.