Mean / Individual Confidence Interval in Mixed Models for new data points

[#1]

Hi everyone,

I fit a mixed model with one random effect in JMP 17. I would like to get confidence intervals for new predictions, i.e. data points which where not used in the model fitting. However, if I go to the red triangle in the fit model platform and use "Save Columns -> Mean/Individual Confidence Interval", it just gives values for the data points used in the model fitting. Is there some way in JMP or JSL to get confidence intervals for new data points? 

If I use models for predictions I am also interested in the uncertainty of this prediction. 

I would appreciate any help!

Best,

Anja

[#2]

Hi Anja: If you add new rows to your data set, with the data point values you want to predict at, but leave the "Y" as missing ( . ).  Then it will give you what you are looking for.

[#3]

Thanks for your quick reply.

Unfortunately, it doesn't. That is exactly what I tried. It just gives the Confidence Intervals for data points where "Y" is not missing and was used in the model fitting. 

What version of JMP do you use? I use JMP Pro version 17.

[#4 · Accepted solution]

Hmm. Are you sure you have all of your predictors in the added columns (with Y missing)?  That means values for all factors (fixed, random, whatever) that are in your model (categorical and numerical), and any "by" variable you may have used. I use JMP V17, but the fact that you use Pro should not matter.

[#5]

Thank you very much. I forgot to add a value to the random factor since the prediction is supposed to be for a new group which I don't know yet. It works if I insert an already existing level of the random factor but actually does not depend on it.

[#6]

And @Anja_W  ...you do have to be very careful here about what exactly you want. Do you want intervals for a certain combination of factor levels (random and fixed) that you used to build your model? Or, do you want intervals that are based on total variability of the data, and not for any particular level of the random effect(s)? If you want the former, then it should work as I described earlier. If you want the latter, then that is an entirely different animal and I'll point you here.  

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6916346/

[#7]

I have been following this conversation with interest. As far as I can see prediction intervals for both the situations you describe can be calculated in JMP. If you want a prediction interval for a certain level of a random effect you choose

Save Columns --> Conditional Indiv CI

If you want a prediction interval where the random effect variability is included you choose 

Save Columns --> Indiv Confidence Interval

[#8]

Exactly.

But also for the latter "Indiv Confidence Interval" you need to provide a value for the random factor. This is what puzzled me. But the output will be the same no matter what value is provided.

[#9]

@Anja_W  and  @StefanC . Points taken; it does appear this version of JMP has that option, which is an improvement (and perhaps I missed it in previous versions). That said, there is precious little information (that I can find) wrt how JMP actually calculates this "Indiv Confidence Interval". And the research in this area indicates that many of the "standard" methods out there for such intervals are incomplete and in need of improvement; for a summary of the issue relative to JMP and other software, see section 3.5 here ( https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6916346/   ). Therein, wrt JMP, it says:

"In JMP, the “Fit Model” platform allows to run a mixed model and the PIs can be obtained from the “individual CI” option in the “Save Columns” menu. There is no formula and no discussion in the help regarding the PI and its degrees of freedom. One can actually notice that the degrees of freedom for fixed effects (ie, Kenward-Roger) are used for the PI. This is not appropriate as explained in Section 3.2 with PIs too wide (see next section)."

[#10]

I've dug into this further. I calculated prediction intervals using the examples in sections 5.2 and 5.3  (in the link I provided, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6916346/) via JMP and via the method outlined therein. And, much to my surprise...they match! This is a wonderful improvement indeed. Thanks JMP for this update, and thanks @Anja_W  and  @StefanC  for working together to tease this out.