Effect of order of grouping on variance components - Variability Gauge Analysis

[#1]

When performing Variability Gauge analysis I noticed that variance components (main effects) values for each component depend on the order the grouping was performed. 

I’d like to know why we’re seeing these differences to better understand how and in what order we should group our data to get the most accurate representation of variation present.

I've attached the data table with 2 scripts for the 2 different orderings.

[#2]

Hi @Agustin,

You may find the solution here : Launch the Variability/Attribute Gauge Chart Platform (jmp.com)

• X, Grouping: Specify the classification columns that group the measurements. If the factors form a nested hierarchy, specify the higher terms first. If you are doing a gauge study, specify the operator first and then the part.

In clear : if you have all interactions done between your factors in your study, then order is not so important. If there is a hierarchy or nested situation, then you should go from general to specific factors.

You can see an example in the topic here : Solved: Re: Gage R&R: Nested vs Crossed vs Nested Then Crossed vs Crossed then Nested - JMP User Com...

where the order of the grouping terms was : Location, then Operator. Part was in "Part, Sample ID" (see the great summary graph by @statman).
In your example it is a bit more difficult as the name of the variables or the design and scope/goal of the study is not known.

Hope this help you,

[#3]

Thank you for your reply. In our case the variables are: Site, Operator, Lot, Instrument and Run.

There are 3 sites, each with different Operators, Instruments and Runs. And there are 3 lots, each site has tested 2 lots.

So I guess Site < Operator < Instrument < Run  are nested and lots are crossed? However I can't seem to do nested then crossed.

What are your thoughts? Both the analysis in the order Site < Operator < Instrument < Run < Lot for main effects and for nested give "reasonable values" but I don't know if main or nested is the more representative.

Thank you

[#4]

If you could map/draw the situation it would be more clear, as I don't get some parts :

- Do operators use several instruments in the corresponding site (and so, for each site, do you have all combinations between operator and instruments ?) ?
- Are the 3 lots the parts to inspect ? Are they the same used and inspected in the different sites ? Only 2 of 3 lot is tested in each site (and is the attribution by site random ?) ?

I don't get why run is part of the grouping variables ? Runs are unique ID/values ?

From your screenshots and table you seem to have confounding between variables, so the safest way to analyze it for me would be to look at main effects only. If you have a clear structure Nested/Crossed or Crossed/Nested, then you may go into more details with this type of model. 

[#5]

Here is the diagram outlining all the tests.

Site 1 has 2 instruments and they are used as shown, sites 2 and 3 have only 1 instrument.

Lot refers to a manufacturing lot, Site 1 uses lots 1 and 2, Site 2 uses 1 and 3, Site 3 uses 2 and 3.

The reason I have "run" in the grouping variables is because I would like to know how much of the variability is attributed to run-to-run variation.

Hope it's clearer now, thank you 

[#6 · Accepted solution]

It is clearer, thanks.
Operators are nested in site, and Instruments are Crossed with Operators (for each site).
The problem to have a full "Nested then Crossed" model is that not all instruments (and Operators) are measuring the same parts : some 1 and 2, some 1 and 3, and on last site is 3 and 2.

So the safest way will be to look at main effects only in my opinion (perhaps experts in this forum would have other ideas).

Looking at your variable "run", I'm not sure it makes sense here, since in this run-to-run variability you'll look at different effects combined (site, operators, instrument, part or a mix of some of these factors).
If you want to estimate repeatability, this is already done in the platform when looking at Within Variance ("Within" in your platform "Variance Components" screenshot, and also seen in the graphs above (several points for the same conditions))  : About the Gauge R&R Method (jmp.com)   

[#7]

Thanks a lot for your input!

[#8]

Just to add to Victor's explanation more generally, every analysis of data is dependent on how the data was acquired.  What questions you can answer, what tools should be used for analysis, conclusions from the analysis, confidence in the ability to extrapolate the conclusions, etc. all depend on how the data was acquired.

If there is a hierarchy (nested) relationship between the columns, that hierarchy must be respected in the analysis.  The order of the terms in the model is important (Sequential vs. Partial SS).  If the columns are crossed, there is no hierarchy and interactions between the columns can be estimated.

For your explanation, I also am a bit confused...Each component you listed needs to be better described. Typically, Operator, lot, instrument and run will be nested in site.  Unless you are using the same lots across sites?  Are these incoming lots of material/parts or are they being manufactured at each site? Additionally are instruments confounded with operators (each operator haas their on instrument) or crossed (every operator uses every instrument)?

I recommend drawing a tree diagram that describes how the samples were obtained and measured.  Use coding of the values for each component, but be careful the actual numbers you use (is every number unique or are there repeated patterns in the component.  For example (not for your specific example):

 

 

[#9]

Thank you for your reply. Exactly the same lots are used across sites, it's a manufacturing lot independent of site. At the sites where there is more than one instrument operators use all instruments. 

I think my tree diagram should reflect this, apologies if not.