Setting up a Nested and Crossed MSA Gauge R&R Analysis

[#1]

I have a Gauge R&R study that I have questions about analyzing in JMP Pro 17.1.  First I’ll give the full background and then I’ll ask the questions.  (This is a long post)

Background and Setup:

In this study we have 2 operators and 3 machines (called BuildMachine), each of which produces 3 parts for a total of 9 parts.  Each operator measures all 9 parts three times.

This means (I believe) that the parts are nested under each BuildMachine.  There are 2 replicate runs.  This means we have a total of 54 runs (3 BuildMachines * 3 Parts * 2 Operators * 3 measurements).

To summarize using the button setup in DOE > Special Purpose > MSA Design:

We Nest Part within BuildMachine and then cross both with Operator

The final experimental setup as shown in the MSA design is then:

I wasn’t 100% sure how to set up the analysis in Analyze > Quality and Process > Variability/Attribute Gauge Chart so I reasoned that I will Simulate Reponses in the MSA Design window > Make Table >  Gauge R&R script and then see how JMP set it up for me.

Here is the result:

The Model Type was set to ‘Decide Later’ so I set it manually to Nested then Crossed as than was how I set up the problem in the MDA Design platform.  Is this right?

Questions:

• Did I set up the MSA Design platform correctly with regards to Nesting and Crossed?
• Did JMP set up the Variability/Attribute Gauge Chart window correctly for me? (Especially with regards to putting all of the factors in the ‘X, Grouping’ window?
• Did I pick the correct Model Type in the Variability/Attribute Gauge Chart window? (Nested then Crossed)
• The engineers would like to quantify the difference between the 3 BuildMachines.  I can use ‘Heterogeneity of Variance Tests’ to look for unequal variances.  I didn’t see any options to test for significant differences among the means.  Is the best route to do a Means/Anova test outside of this platform?

Thanks,

Larry

[#2 · Accepted solution]

Hey @LEP !

In terms of how you set up the design, you're spot on! I'm glad to see those "easy buttons" were useful in setting up the design for you. 

Regarding the analysis, you did pick the correct model. However, it looks like the ordering of the factors in the grouping section does matter and it is not correct by default. The Operator factor should be at the bottom of the list with BuildMachine at the top. I found this by using the Heterogenity of Variance Tests and making sure that the nesting structure was correct. It seems that the platform treats Nested then Crossed as first factor nested within second, then both crossed with third. I'll be opening an internal ticket to address this issue for JMP 18. Thankfully, it's easy to fix interactively. Just click and drag Operator to the bottom in the X-Grouping list box. 

For testing the differences in means, I'm inclined to think that you may need a different platform. The EMP table script opens the EMP Measurement Systems Analysis platform, which has a Bias Comparisons red triangle menu item (for the second red triangle in the Measurement Systems Analysis for Y outline). This performs a graphical comparison, but necessarily a formal hypothesis test. I'm not sure if the script has the right structure for the analysis there either, so I'll be including it in my internal ticket. The other option is to run the Fit Model script, with Operator and Part[BuildMachine] as random effects (Attribute>>Random Effect) and BuildMachine as a fixed effect. You can include all the crossed terms as random effects as well. 

A quick note. If the variance component for BuildMachine is a significant contributor to the overall variation, then that in itself would indicate significant differences among the machines. But if you're looking to quantify what those differences are, then you can see my previous suggestions. 

Hope that helps!

Caleb

[#3]

Hi  (Caleb), 

Thanks for your response!

I watched your ‘Designing and Evaluating Measurement Systems Studies’ video, it was excellent!

I do have a few follow-up questions:

• In Figure A below (original Gauge R&R script ordering), it easy to visualize the variability between the two operators as a whole.  After the reordering (B) however, the operators are mixed under each part making it harder to compare them. (Note: I also looked at the ‘Variance Components for Gauge R&R tables and they are different for A vs B so based on your input B must be the correct one?).  So for at least the Variability plot I like the original ordering.  Based on this would you recommend I run it both ways: once for the variability chart and once (with your suggested order) for the Heterogeneity of Variance Tests and the Variance Components for Gauge R&R table?

Figure 1 – Comparison of parameter orders

• I also ran the original ‘EMP Measurement Systems Analysis Script’ and that script orders the plots the way I prefer (like Figure 1A with all parts grouped under each Operator).  Is the EMP script possibly mis-ordered as well?

Figure 2 – Plot from unmodified EMP script

• Finally. I like your Bias Comparisons idea.  I have a question about your Fit Model suggestion but I need to think it through more first before I ask.

Thank you so much for your assistance!

-Larry

[#4]

Hi @calking !

I forgot to tag my reply to you last week.  Can you check out my reply where I have some follow-up questions?  Specifically I'm not sure now that the automatically generated script for EMP is correct for my 'Nested then Crossed' case either.

Thanks,

Larry

[#5]

Hey @LEP !

No worries! 

So I just met with the developer of the Variability Chart/Gauge R&R platform and just learned something interesting about this particular model! As it turns out, the way the terms are originally ordered is correct...for some parts of the platform. 

If you're interested in AIAG metrics and/or the variability chart, you want the original ordering of the factors. This is because the last factor is assumed to be the Part factor, and in the default ordering used by MSA Design, this would be the case. However, if you wish to run other options, such as the Variance Components, then the new order should be used to get the correct structure. This strange behavior is because the Nested then Crossed option was not really built to handle a structure where Part is nested in one factor, than both crossed with another factor. Apparently the AIAG did not consider this type of structure in their methodology. 

If you do need to stick with this structure, then you might be best served using the Fit Model (or Fit Mixed) table script, which is very flexible when it comes to different structures. You'll also want to consider @statman 's questions to make sure this is indeed the structure that will achieve your goals. 

Hope that helps!

[#6]

@calking - Thanks for getting back with me.   

For a whole host of reasons this is the structure that I am stuck with so I have to make do with what I have.

Since we last talked, I have read "EMP: A Better Way to Do R&R Studies" and "An Honest Gauge R&R Study" both by Donald J. Wheeler so I am much more interested in the EMP method now.

Question: For the EMP script, can I assume I need to change my variable order based on if I am looking at the Average and Range charts (keep as as) or  if I want the 'EMP Gauge R&R Results' and 'Variance Components' tables (move operator to the bottom) ?

[#7]

Hi @LEP ,

I wanted to try to answer your last question about the EMP script.  If you move the operator to the bottom of the list for the Variability platform (or assign the operator as the "part" in the EMP MSA dialog), this model will fit the design and situation you have described.  However, it will be treating your "Operator" as the "part" or "product". 

I hope this helps.

[#8]

Thank you @Laura_Lancaster !

[#9]

Larry, Excuse my bias, but before I respond to your technical questions, I have some questions/comments:

1. A study of the measurement process requires you to consider what you are assessing and what comparisons you need to make.  An important question to ask is:  What do you want your measurement system to be capable of?  What do you need it to consistently quantify?  These are important questions and have a huge effect on how you select your samples (parts) for the study.

2. Are 3 parts (I assume randomly sampled?) a representative sample of the variation you want your measurement system to discriminate?  (BTW, a random sample likely increases the variation in the samples to be measured.)  Do you feel confident that this plan will provide enough evidence in the consistency of the measurement process?

For example, you may want your measurement system to be able to specifically measure within part/sample variation (or part-to-part,  lot-to-lot, etc.).  If this is not part of how you collect your data, you cannot determine if it can.

[#10]

@statman 

Thank you for your cpmments statman . Unfortunately,  for a whole host of reasons (many goals and constraints besides those I've mentioned) this is the structure and data quantity that I am stuck with so I have to make do with what I have.