topic Prediction Profiler - Upper and Lower Limits - how to compute? in Discussions

Prediction Profiler - Upper and Lower Limits - how to compute?

BlockPeccary466 — Sat, 10 Jun 2023 23:56:44 GMT

Hi experts,

I'm trying to figure out how the upper and lower limits are computed on the prediction profiler for a categorical regression model.

Here is the model and a profiler snapshot for (White, Low Density, Night):

To get the mean response of 0.305017, we can simply add up

0.4650093 + (-0.03985) + (-0.0513424) + (-0.0687996) = 0.305017

But how are the lower/upper limits of 0.22501 and 0.38502 calculated?

Originally, I simply summed up the corresponding standard deviations to find a standard deviation for the response so -

0.0269889 + 0.0243968 + 0.0126292 + 0.0126749 = 0.0766898

then multiplied by z = 1.96

0.0766898 x 1.96 = 0.150312008

but from the profiler, this value is much lower:

0.38502 - 0.305017 = 0.080003

I also tried to look at the formula for the upper and lower limits but for some reason could not get it to show (I could get the predicted response formula to show).

If anyone could point me in the right direction I would be very grateful, thank you!

Re: Prediction Profiler - Upper and Lower Limits - how to compute?

Mark_Bailey — Wed, 09 Nov 2022 19:37:52 GMT

Those limits are the confidence bounds on the predicted mean response. Any textbook about ordinary least squares regression will explain the origin and computation of these limits. Here is the matrix form for the computation:

X0 is the vector of factor levels. X is the model matrix for the regression. Sigma is the root mean square error from the regression. The square root of the variance is the standard error. Multiply the standard error by the appropriate quantile from the t distribution.

Re: Prediction Profiler - Upper and Lower Limits - how to compute?

BlockPeccary466 — Wed, 09 Nov 2022 21:23:04 GMT

Thanks for the response, Mark, it's truly appreciated!

I understand that the CI is calculated from the data, but I should have mentioned that we don't actually have the original data. The model is from a project from several years ago so we're still trying to locate the data. Hopefully we can locate the set and calculate the CIs.

I was wondering if there's any way to estimate the CIs based on the standard deviations of the model coefficients? Right now those screenshots are all we have to work with so I thought there might be a way to get the CIs just based on the effects table.

I truly appreciate the help - I've done a lot of searching trying to figure this out but it looks like we might be out of luck till we find the data.

Re: Prediction Profiler - Upper and Lower Limits - how to compute?

Mark_Bailey — Thu, 10 Nov 2022 14:46:00 GMT

You cannot use the standard error of the parameter estimate unless you know that all the columns in the model matrix X are uncorrelated. The diagonal elements of X`X are the square of the standard errors, so you could construct a diagonal matrix from those, but the off-diagonal elements are important in this calculation. They can contribute a lot if they are not equal to zero. These elements might be available if the Correlation of Estimates table appears in any screenshots.

Re: Prediction Profiler - Upper and Lower Limits - how to compute?

BlockPeccary466 — Mon, 14 Nov 2022 14:27:29 GMT

Thanks Mark - really appreciate your expertise in this matter, this is great information - we'll keep hunting and hopefully find it out!