Why different results for variance components in Variability Gauge Analysis Report and in Fit Model Analysis Report?

[#1]

Hi all, 

could someone please explain why the two reports mentioned above seem to give different results for the F-Ratios of the main effects of operator and part, but give consistent results for the F-Ratio of the interaction effect operator*part? 

Data file: Variability Data/2 Factors Crossed.jmp   (JMP example data file. Use >Help>Sample Data Folder in JMP to get it) 

Test #1:

I did the JMP example
https://www.jmp.com/support/help/en/19.0/index.shtml#page/jmp/example-of-a-variability-chart.shtml#w...

In the Variability Gauge Analysis Report the variance componentes were displayed using
> Variability Gauge Analysis for Measurement >  Variance Components

Attached file "Variance Components from Variability Gauge Analysis Report.png" shows the result I obtained. 

Test #2:

I did a similar calculation using "Fit Model" with model terms  Operator, part#, and Operator*part#.
(see attachment:  Model Specification used for Fit Model.png )

Attachment "Variance Components from Fit Model Report.png" shows a screeshot from the result obained for "Analysis of Variance" and "Effect Test".

I expected to confirm the results of Test #1.  

Most ANOVA results were consistent in both reports  -  marked with green rectangles in both pictures.

However, some results for the F Ratios were different in both reports  -  marked with red rectangles in both pictures.

Description of the observed difference

I remember that ANOVA calculates F Ratio for each model term by dividing the Mean Square (MS) of this model term by the MS Error, which seems to be consistent with F Ratio results shown for all model terms in the Fit Model Report. 

F Ratio = MS (term) / MS (Error)

Assuming that the term MS(Within) corresponds to the term MS(Error), the Variability Gauge Analysis Report Report this seems to apply an analogous formula to compute the F ratio of the interaction term Operator*part#.

F Ratio(Operator*part#) = MS (Operator*part#) / MS (Within)  = 0.02087039 / 0.00413887  =   5,04253335  

But, I was not able to re-produce the values shown for the F Ratios of the main effects of Operator and of part# using MS(Within).

MS(Operator) / MS(Within) = 0.0274444  / 0.00413887  =  6.6308920  (not equal to 1.3150)
MS(part#)      /   MS(Within) = 0.2926204  / 0.00413887  =  70.7005535  (not equal to 14.0209)

The Variability Gauge Analysis Report seems to use a similar formula to compute the F Ratios for the main effects of Operator and of part#, however using MS(Operator*part#) instead of MS(Within).

MS(Operator) / MS(Operator*part#) =  0.0274444  /  0.02087039  =  1.3149922  (similar to F Ratio given)

MS(part#) / MS(Operator*part#)  =  0.2926204  /  0.02087039  =  14.0208401  (similar to F Ratio given)

In the attachment you will also find a copy of the JMP example data file with two scripts to re-produce the above tests I did.

Please, could somebody comment on this discrepancy and explain why F Ratios are calculated this way in Gauge R&R ?

Thank you very much for helping. 

[#2]

Hi @DOEMonkey : Good question, and welcome to the community! I'll take your question in two parts:

1. Please, could somebody comment on this discrepancy?

The discrepancy comes from the fact that it your Fit Model platform you haven't designated all of the effects as Random (as is assumed in the Variability Gauge Analysis). If you do that and choose EMS(Traditional) for Method you will get the same results.

2. Explain why F Ratios are calculated this way in Gauge R&R .

See page 6 here. https://homepages.math.uic.edu/~wangjing/stat481/Stat481-LectureNote-10.pdf

As you can see, under null hypothesis (H0) the appropriate error term for A and B is MSAB (i.e., makes the respective F ratio =1).

Come back with any questions.