JMP 11 and 10 has a great module entitled Quality and Process. In one option (i.e. measurement system analysis) it is offering EMP results which include intraclass Correlation coefficient
QUESTION: Can this ICC coefficient of EMP results be used for evaluating the appropriateness of a Multilevel Model (HLM) or Mixed Model procedure???
Or it can be used only in situations of inter-rater reliability???
The model used in the Measurement Systems Analysis EMP platform is a random effects model that is specifically tailored to measurement systems analysis where you are measuring something, which we refer to as a part. The ICC shown in the EMP results gives the proportion of variation in your measurements that can be attributed to variation in the parts. The only case in which you could view these EMP models as a simple multilevel model is when you have a part variable but no grouping variables.
If you would like to calculate an intraclass correlation coefficient for a multilevel model, it is possible to do this for simple multilevel models using REML in JMP or for more complicated multilevel models in JMP Pro version 11. The intraclass correlations can be easily
computed based on the covariance parameter estimates found in the report.
I hope this helps.
Thank YOU very much for your answer. Is there any online reference material for doing this analysis in JMP following the steps you outlined besides your insightful answer?
Here is a link to some online documentation for REML:
Here is a link to some online documentation for the new Mixed Models platform in JMP Pro 11:
This blog post might also be helpful if you have JMP Pro 11:
I hope that helps.