Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

- JMP User Community
- :
- Discussions
- :
- What does it mean if both my predicted plot P value and my Prob > F are both sig...

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Created:
Jul 31, 2019 11:15 AM
| Last Modified: Aug 1, 2019 6:22 AM
(752 views)

Hi,

I'm running datasets where I'm getting really nice RMSE (0.86-0.90) and P value < 0.0001 for my predicted plots, but my Prob > F is still signficant for Lack of Fit. What does this mean? Example pasted below.

1 ACCEPTED SOLUTION

Accepted Solutions

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Created:
Jul 31, 2019 12:10 PM
| Last Modified: Jul 31, 2019 12:13 PM
(742 views)
| Posted in reply to message from JBeierle 07-31-2019

A model can be significant and explain a lot of variability and still exhibit lack of fit. For example, the model may be missing an important term, as shown in the attached simple example.

A significant lack of fit usually indicates that the model form is not correct and usually indicates that there is a curvilinear relationship. Residual plots are the best and easiest tool to use to try and determine why there is a lack of fit. A good residual plot will show random scatter. Any pattern will indicate a systematic problem with your chosen model. For my simple example, here is a residual plot (there is more than one residual plot, so look at all of them to help determine the problem with your model).

There is an obvious pattern here indicating that I need a quadratic term in the model to estimate the curvature. I THINK this is a problem with your model as well, but the residual plots should help you.

Dan Obermiller

2 REPLIES 2

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: What does it mean if both my predicted plot P value and my Prob > F are both signficant?

*sorry for the typo* - Rsq are really nice. NOT RMSE.

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Created:
Jul 31, 2019 12:10 PM
| Last Modified: Jul 31, 2019 12:13 PM
(743 views)
| Posted in reply to message from JBeierle 07-31-2019

A model can be significant and explain a lot of variability and still exhibit lack of fit. For example, the model may be missing an important term, as shown in the attached simple example.

A significant lack of fit usually indicates that the model form is not correct and usually indicates that there is a curvilinear relationship. Residual plots are the best and easiest tool to use to try and determine why there is a lack of fit. A good residual plot will show random scatter. Any pattern will indicate a systematic problem with your chosen model. For my simple example, here is a residual plot (there is more than one residual plot, so look at all of them to help determine the problem with your model).

There is an obvious pattern here indicating that I need a quadratic term in the model to estimate the curvature. I THINK this is a problem with your model as well, but the residual plots should help you.

Dan Obermiller

Article Labels

There are no labels assigned to this post.