Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Re: goodness of prediction

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Sep 18, 2018 7:44 AM
(5060 views)

Dear JMP experts,

would you advice me where can I find a Q square to compare goodness of prediction of two models constructed?

Eventually, What indicator do you use instead?

What does the "predicted RMSE" value under the graph in actual by predicted plot means?

thank you very much,

Michal

2 ACCEPTED SOLUTIONS

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I try NOT to use a single statistic or visualization to answer the question "Which model predicts best?" That's falling victim to a dreade disease one of my statistics professors told me about called "mononumerosis".

But I look at many different statistics and visualizations...if I had to pick some must see visualizations then all roads lead to various types of residuals plots. JMP supports all the usual suspects so I suggest starting there. After all, these plots tell you alot about variation and performance that is explained and unexplained. And at the end of the day, I encourage you to consider the model that helps you solve your practical problem...so domain expertise is equally important if not more so, than the statistics and visualizations.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I tend to agree with Peter's assessment. Understanding what makes a good model is like putting together a jigsaw puzzle. No single piece gives you the entire picture.

I would tell you where to find "Q Square", but honestly, I have never heard of that statistic. JMP offers several other statistics to help you assess the fit of a model like RSquare, RSquare Adjusted, AICc, BIC, etc. But again, no single number is really sufficient.

Finally, underneath the actual by predicted plot, JMP displays RMSE. It is not a "predicted RMSE". The predicted part is completing the labeling of the X-axis variable. Enclosed is a simple example (from a bad model) where they were trying to predict Claim Amount. Notice the labeling of the X-axis is "Claim Amount Predicted". The RMSE is just that: the Root Mean Square Error which you can see in the Summary of Fit table.

Dan Obermiller

4 REPLIES 4

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I try NOT to use a single statistic or visualization to answer the question "Which model predicts best?" That's falling victim to a dreade disease one of my statistics professors told me about called "mononumerosis".

But I look at many different statistics and visualizations...if I had to pick some must see visualizations then all roads lead to various types of residuals plots. JMP supports all the usual suspects so I suggest starting there. After all, these plots tell you alot about variation and performance that is explained and unexplained. And at the end of the day, I encourage you to consider the model that helps you solve your practical problem...so domain expertise is equally important if not more so, than the statistics and visualizations.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: goodness of prediction

Thank you very much for your answers! BR, Michal

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I tend to agree with Peter's assessment. Understanding what makes a good model is like putting together a jigsaw puzzle. No single piece gives you the entire picture.

I would tell you where to find "Q Square", but honestly, I have never heard of that statistic. JMP offers several other statistics to help you assess the fit of a model like RSquare, RSquare Adjusted, AICc, BIC, etc. But again, no single number is really sufficient.

Finally, underneath the actual by predicted plot, JMP displays RMSE. It is not a "predicted RMSE". The predicted part is completing the labeling of the X-axis variable. Enclosed is a simple example (from a bad model) where they were trying to predict Claim Amount. Notice the labeling of the X-axis is "Claim Amount Predicted". The RMSE is just that: the Root Mean Square Error which you can see in the Summary of Fit table.

Dan Obermiller

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: goodness of prediction

Thank you very much for the answer, M