Confidence interval in Fit Model Actual by Predicted plot

[#1]

I would like to know how the confidence interval in Fit Model Actual by Predicted plot is calculated. I tried to manually plot Actual vs Predicted with Fit Y by X and the confidence interval is different with the one in Fit Model. I can manually calculate the confidence interval of Fit Y by X but cannot figure out how to calculate the one in Fit Model. Please advise. Thanks a lot!

[#2 · Accepted solution]

Hi @vcl0124 : It is a leverage plot.

https://community.jmp.com/t5/Discussions/how-to-read-actual-by-predicted-plot/td-p/390471

https://www.jmp.com/support/help/en/17.2/index.shtml#page/jmp/statistical-details-for-leverage-plots...

[#3]

Hi MRB3855,

Thanks a lot for your advice! However, I found the z^2 in below equation should be (z - mean of predicted)^2. Do you agree?

Another question is about fit model with (1st plot below) and without (2nd plot below) interception. I believe the with interception one is the better model but the leverage plot is wider. How to understand this? Please advise. Thanks a lot!

[#4 · Accepted solution]

Hi @vcl0124 :

Q:I found the z^2 in below equation should be (z - mean of predicted)^2. Do you agree?

A: not according to Sall (1990). See JMP reference 

https://www.jstor.org/stable/2684358

https://blogs.sas.com/content/iml/2022/07/11/confidence-bands-partial-leverage-plot.html

Q: I believe the interception one is the better model but the leverage plot is wider. How to understand this? Please advise.

A: This may be a bit involved for this forum; but , briefly, unless you have a really good reason to exclude the intercept, do not exclude it. The second plot you show, clearly shows the model is not correct so any inference drawn from that is not correct as well.

Here is a pretty good discussion(s). 

https://stats.stackexchange.com/questions/7948/when-is-it-ok-to-remove-the-intercept-in-a-linear-reg...

https://communities.sas.com/t5/SAS-Communities-Library/The-Why-How-and-Cautions-of-Regression-Withou...

[#5]

Hi MRB3855,

Thanks a lot for your advice!

For z^2 or (z - mean of predicted)^2, I cannot read the paper. However, the example in the 2nd link seems to have mean of predicted = 0. Actually, if you look at the shape of the leverage plot of the with interception case, the confidence interval is the smallest at mean of predicted. Therefore, z^2 must be smallest at mean of predicted and that's why I suspected the z is shifted by mean of predicted (replace z by z - mean of predicted).

For the leverage plot of without interception, I figured out it is tested against y=0 instead of y=mean of y and that's why the confidence interval is narrower as y=0 is very unlikely. I actually think it is OK to test without interception model against y=mean which should give us more meaningful information about whether the model is valid, am I correct?