How to interpret the result of random effects model in JMP?

[#1]

I fitted a random effects model with 2 factors (Part No. and Operator) by selecting the "Random Effects" option under "Attributes" on "Fit Model" window (with interaction Part*Operator).

I received the following output under REML Variance Components Estimates table.

Random Effect	Var Ratio	Var Component	Std Error	95% Lower	95% Upper	Wald p-Value	Pct of Total
Part	1.1555556	1.7333333	0.8656795	0.0366326	3.430034	0.0453*	53.608
Operator	-0.004115	-0.006173	0.0218	-0.0489	0.0365544	0.7771	0.000
Part*Operator	-0.199588	-0.299383	0.1464372	-0.586394	-0.012371	0.0409*	0.000
Residual		1.5	0.3354102	1.0110933	2.4556912		46.392
Total		3.2333333	0.9283863	1.9759148	6.2317667		100.000

  -2 LogLikelihood = 203.88510393

How to interpret these results?

[#2]

The main result of REML is the Var Component column. The components from the model sum to the Total. The next result is the confidence interval or p-Value for the Wald ratio. The Part and Part*Operator estimates appear to be significantly different from zero. Model hierarchy requires keeping the Operator term even though it does not appear to be different from zero.

The negative estimate for the interaction component is because it is really a covariance, which can be negative. Another community member might have more insight in the interpretation in the case of MSA.

The Part variance is about 54% of the total and the residual (repeatability) is about 46% of the total, so this measurement system is not good.

Did you also analyze your data with the Variability Chart platform?

[#3]

I stumbled on this discussion due to trying to understand how to interpret negative variance component estimates with a low Wald p-value (although in my case the negative variance component is for a main effect).  I was just assuming that a low Wald p-value should only be considered truly important if the variance component estimate was positive, if the estimate is negative it doesn't make sense to say it is statistically significant.  But your comment about the negative estimate for the interaction has me confused.  I would interpret the interaction variance component as the variation due to the interaction between parts and operators, so if the estimate is negative I was thinking it would be interpreted as not truly significant, regardless of the Wald p-value.  Thanks.

[#4]

This is not an easy discussion technically, but practically I have some thoughts:

1. We really don't care as much about the actual values of the variance components, but the relative values compared to the others in the study. 

2. It is the magnitude of those components that is of most interest (not the sign).  Usually JMP will switch to Bayesian estimates when there are negative REML components.

3. Always be cautious of p-values.. they are a function of comparisons within the data set collected. Whether they have any association with reality (or are useful) is a function of how the data was collected.

4. In addition, unusual data points can have a huge effect on variance component estimates and subsequent quantitative tests.  ALWAYs plot the data and evaluate the data for unusual or special cause data points.

5. Plot the data compare to your predictions...does it make sense?  Can you explain what you see?  Don't turn off engineering or science.

[#5]

I'll add just a bit to @Mark_Bailey 's explanation by making sure that you ALSO plotted the data using a variety of the graphical techniques available in the Gauge R & R and/or Measurement Systems Analysis platforms and subplatforms. These charts can often times be very informative by visualizing features/characteristics in the measurement system that are of interest and valuable over and above the static tabular results that are also presented. Here's a link to the online documentation for the MSA platform as a good place to start:

https://www.jmp.com/support/help/14-2/measurement-systems-analysis.shtml#

And here's a JMP On Demand Mastering JMP webinar that I hosted during my SAS tenure that you might also find valuable:

https://www.jmp.com/en_us/events/ondemand/mastering-jmp/evaluating-and-monitoring-your-process-using...