Repeatability threshold?

[#1]

Hello everybody,

I have just started to use the Measurement System Analysis and I was wondering if there exists some thresholds regarding the repeatability value or is it something  "data dependent"?

Thanks in advance for your inputs!

[#2 · Accepted solution]

Hi @anne_sa,

There are lot of options for MSA depending on the guideline you may refer to (AIAG, VDA, Bosch, ...).
What is common and critical for MSA is to differentiate part-to-part variation from measurement variation (equipment, operator, other external factors...) which can be calculated with %R&R in JMP (measurement quality), and also the accordance of your measurement variability depending on your specification (customer quality), which is calculated with P/T (Precision vs. Tolerance ratio, noted "Precision to Part variation" in JMP). The formulas are shown in my screenshots.

Both of these indicators should be <30% to have acceptable results (on the second link JMP provides some guidance about how to interpret the values), which means you want a batch variance at least 10 times higher than the rest of the variance (repeatability, operator and operator x batch).

You might find more infos here for JMP: https://www.jmp.com/support/help/en/16.2/index.shtml#page/jmp/about-the-gauge-rr-method.shtml#ww1925...

https://www.jmp.com/support/help/en/16.2/index.shtml#page/jmp/gauge-rr-option.shtml#

And more generally about assessing measurement quality : https://sixsigmastudyguide.com/measurement-systems-analysis/

This is also often described as "Type-II study", where you assess operator, equipment and measurement variation. But prior to this study, it is highly recommended to realize "Type-I study" where you keep one operator and one equipment, just to check measurement variability and that your measurement process is statistically in control and stable (assessment done with the calculation of Cp/Pp and Cpk/Ppk, and through the use of control chart).

I hope this first answer will help you, 

[#3]

Hopefully I have interpreted your question correctly...There is no universally accepted threshold value as the amount of variation in the precision repeatability of the measurement system that is acceptable is situation dependent.  Also, recognize, this is only one component of the measurement system variance and adequacy.  Is the measurement system precision repeatability consistent?  Is there adequate discrimination (effective resolution) which can impact estimates of precision? How much of the total amount of variation, that is of engineering or scientific significance, does the repeatability account for?  I believe this is a better question than comparing to possibly arbitrary specifications.

[#4]

Thanks @Victor_G and @statman for your answers, that's really helpful

[#5]

Glad you find these answers useful @anne_sa !

As I was mentioning, acceptance criteria and decision threshold may differ depending on the MSA guideline used, but they are quite similar in the process and evaluation of the measurement system. There might be some checks you would like to run before Type II and Type I studies, as also mentioned by @statman : resolution, accuracy/bias, linearity, stability of measurements.
For Type I and Type II studies, the selection of the product and corresponding batches (for Type II) is very important : for Type I, the product chosen should be representative of the future samples measured by the equipment, and for Type II the batches should all be conform and cover the specification range (from LSL to USL in order to be able to see some part-to-part variation, meaning your equipment is able to differentiate quite precisely your batches).

Also worth to mention, not all equipment may pass the %R&R acceptance value even if correctly managed and state-of-the-art. 
But with domain expertise and scientific reasoning, you will be able to conclude from your data if the precision from your equipment could be improved, or if there is no possible improvement.

Don't hesitate if you have further questions or if you would like to know more

[#6]

Just a bit of another point of view regarding some of your comments, Victor.  Particularly:

For Type I and Type II studies, the selection of the product and corresponding batches (for Type II) is very important : for Type I, the product chosen should be representative of the future samples measured by the equipment, and for Type II the batches should all be conform and cover the specification range (from LSL to USL in order to be able to see some part-to-part variation, meaning your equipment is able to differentiate quite precisely your batches).

I agree completely the selection of "samples" for your study is of particular importance, however that section depends on what questions you are trying to answer.  For example, if you want to be able to detect within batch variation, you will need multiple measures within batch for your study.  Limiting samples to just the spec limit range may not adequately describe measurement system capability (Don't assume the specs have anything to do with reality as they are ofter derived independent of actual variation). If you want to understand the adequacy of the measurement system for providing insight to your hypotheses, the samples should be collected as a function of your hypotheses.

Recognize conclusions regarding measurement systems capability are conditional upon how the study is performed and what comparisons are being made.  Change either of those, so goes the conclusions.  Measurement studies are seldom a one shot event.

[#7]

Hi @statman,

Thanks a lot for your comprehensive and complementary answer.
There are a lot of recommendations in these guidelines that an answer or a post can't hopefully resume, so that's why I may have made some shortcuts to focus on the main essential points to have in mind. For sure, the business problem/question, domain expertise and appropriate selection method remain at the heart of any analysis topic.
My aim was more to provide tools, options and evaluation methods to run this MSA (as stated in the original question about any threshold that may help guide decision), not provide a full and complete method.

For type II studies, you are able to evaluate repeatability as well as other sources of variation, since MSA guidelines require a minimum number of runs in total AND minimum number of runs per factors, so you'll have enough data to evaluate within variation. According to VDA MSA, for k appraisers (k >=2), n parts to measure (n >= 5, often 10) and r number of measurement per condition and appraisers (r >=2), you need to have k.r.n >=30 experiments in the randomized design to gain an understanding about your equipment performance.
I never consider an MSA study a one shot event, but more like a starting point to build reliable equipment assessment and a real strategy about data monitoring.

I hope my answer will dismiss any misunderstanding

All the best,