Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Re: ERROR: 0.05 significant Curve shaded error plot even though my model is sign...

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Apr 22, 2019 12:05 PM
(3953 views)

I fit model in JMP. Everything but my graph looks very strange.

THe model is significant at 0.0476

RSquare 0.951662

RSquare Adj 0.794562

Root Mean Square Error 0.928013

Mean of Response 88.68333

Observations (or Sum Wgts) 18

The pink area seems to cover everything as an error. Has anyone else Encounter this?

1 ACCEPTED SOLUTION

Accepted Solutions

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Ah, the mixture situation explains alot of this. You are fitting a Scheffe model. They will typically have more error associated with the prediction because of the built-in collinearity of the mixture factors.

Expand your y-axis scale on the observed versus predicted plot. I think you will see the error bounds appear. There will likely be high error because you really do not have many significant terms in this model. Perhaps only one term, X6.

The significance of your X1-X4 terms is misleading. The p-values are associated with the test comparing the parameter estimate to 0. But you are fitting a Scheffe model. The parameter estimate for a Scheffe mixture term is the combination of the mean of the data AND the effect of the component. So a more meaningful test would be to compare the parameter to the overall mean (88.68 for your scenario). Considering that, you will see that none of the X1 through X4 terms look to be "significant". You can verify by adding the 95% confidence interval on for each of the parameter estimates. If the confidence intervals for X1 - X4 contain the overall mean, that component is not having much of an effect.

Stated another way, it looks like you are getting about the same response everywhere within the mixture space. X5 does not seem to have an impact, but X6 does.

Dan Obermiller

5 REPLIES 5

Highlighted
##
Re: ERROR: 0.05 significant Curve shaded error plot even though my model is significant with high R2 value

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I think a few more details might be needed. Typically, these strange types of results occur because of very high multi-collinearity in the model terms.

What do the leverage plots look like?

If you right-click on the Parameter Estimates table and add the VIF column, what do those numbers look like?

Dan Obermiller

Highlighted
##
Re: ERROR: 0.05 significant Curve shaded error plot even though my model is significant with high R2 value

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Hi Dan, Thank you for your quick response.

The Leverage Plots all looks pretty good. Example Pics attached here.

The VIF Is pretty high (>5):

VIF

12.19448

11.013741

16.435892

28.099305

1.1546501

1.2200138

7.8651502

14.560553

5.9118511

10.052586

8.2589787

5.4652854

5.4652854

1.0590402

However, even when I step thru the model to eliminate some of these factors, a refined version of the model with VIF less than 3 still has the same shaded plot. (attached here). This is also a mixture study so X1, 2, 3, and X4 are inherently correlated (add up to a sum of 100%).

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Ah, the mixture situation explains alot of this. You are fitting a Scheffe model. They will typically have more error associated with the prediction because of the built-in collinearity of the mixture factors.

Expand your y-axis scale on the observed versus predicted plot. I think you will see the error bounds appear. There will likely be high error because you really do not have many significant terms in this model. Perhaps only one term, X6.

The significance of your X1-X4 terms is misleading. The p-values are associated with the test comparing the parameter estimate to 0. But you are fitting a Scheffe model. The parameter estimate for a Scheffe mixture term is the combination of the mean of the data AND the effect of the component. So a more meaningful test would be to compare the parameter to the overall mean (88.68 for your scenario). Considering that, you will see that none of the X1 through X4 terms look to be "significant". You can verify by adding the 95% confidence interval on for each of the parameter estimates. If the confidence intervals for X1 - X4 contain the overall mean, that component is not having much of an effect.

Stated another way, it looks like you are getting about the same response everywhere within the mixture space. X5 does not seem to have an impact, but X6 does.

Dan Obermiller

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Thank you Dan. That makes sense.

Given this interpretation, my model is still significant at alpha <0.05 correct? Even if it is just base on the value of X6?

Thanks again

Highlighted
##
Re: ERROR: 0.05 significant Curve shaded error plot even though my model is significant with high R2 value

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

You are correct that the overall model is likely significant, based on the X6 term.

Realize that because of the issues with the main effects of a Scheffe model, the overall model test may show as significant even if the model is not.

Dan Obermiller