cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
Michal
Level II

goodness of prediction

Dear JMP experts,

would you advice me where can I find a Q square to compare goodness of prediction of two models constructed?

Eventually, What indicator do you use instead?

What does the "predicted RMSE" value under the graph in actual by predicted plot means?

thank you very much,

Michal

2 ACCEPTED SOLUTIONS

Accepted Solutions
Peter_Bartell
Level VIII

Re: goodness of prediction

I try NOT to use a single statistic or visualization to answer the question "Which model predicts best?" That's falling victim to a dreade disease one of my statistics professors told me about called "mononumerosis".

 

But I look at many different statistics and visualizations...if I had to pick some must see visualizations then all roads lead to various types of residuals plots. JMP supports all the usual suspects so I suggest starting there. After all, these plots tell you alot about variation and performance that is explained and unexplained. And at the end of the day, I encourage you to consider the model that helps you solve your practical problem...so domain expertise is equally important if not more so, than the statistics and visualizations.

View solution in original post

Re: goodness of prediction

I tend to agree with Peter's assessment. Understanding what makes a good model is like putting together a jigsaw puzzle. No single piece gives you the entire picture. 

 

I would tell you where to find "Q Square", but honestly, I have never heard of that statistic. JMP offers several other statistics to help you assess the fit of a model like RSquare, RSquare Adjusted, AICc, BIC, etc. But again, no single number is really sufficient.

 

Finally, underneath the actual by predicted plot, JMP displays RMSE. It is not a "predicted RMSE". The predicted part is completing the labeling of the X-axis variable. Enclosed is a simple example (from a bad model) where they were trying to predict Claim Amount. Notice the labeling of the X-axis is "Claim Amount Predicted". The RMSE is just that: the Root Mean Square Error which you can see in the Summary of Fit table.

Capture.PNG

Dan Obermiller

View solution in original post

4 REPLIES 4
Peter_Bartell
Level VIII

Re: goodness of prediction

I try NOT to use a single statistic or visualization to answer the question "Which model predicts best?" That's falling victim to a dreade disease one of my statistics professors told me about called "mononumerosis".

 

But I look at many different statistics and visualizations...if I had to pick some must see visualizations then all roads lead to various types of residuals plots. JMP supports all the usual suspects so I suggest starting there. After all, these plots tell you alot about variation and performance that is explained and unexplained. And at the end of the day, I encourage you to consider the model that helps you solve your practical problem...so domain expertise is equally important if not more so, than the statistics and visualizations.

Michal
Level II

Re: goodness of prediction

Thank you very much for your answers! BR, Michal

Re: goodness of prediction

I tend to agree with Peter's assessment. Understanding what makes a good model is like putting together a jigsaw puzzle. No single piece gives you the entire picture. 

 

I would tell you where to find "Q Square", but honestly, I have never heard of that statistic. JMP offers several other statistics to help you assess the fit of a model like RSquare, RSquare Adjusted, AICc, BIC, etc. But again, no single number is really sufficient.

 

Finally, underneath the actual by predicted plot, JMP displays RMSE. It is not a "predicted RMSE". The predicted part is completing the labeling of the X-axis variable. Enclosed is a simple example (from a bad model) where they were trying to predict Claim Amount. Notice the labeling of the X-axis is "Claim Amount Predicted". The RMSE is just that: the Root Mean Square Error which you can see in the Summary of Fit table.

Capture.PNG

Dan Obermiller
Michal
Level II

Re: goodness of prediction

Thank you very much for the answer, M