Lower Control Limit of Test-Retest Plot in MSA

[#1]

Hello All,

I had a statistical interpretation question. I had run a few MSA studies and was interested in explaining or interpreting the results of the Test-Retest plot. 

The definition that I find on the JMP website is 

"Test-Retest Error

Indicates measurement variation or repeatability (also known as within error or pure error)."

In most cases, my data points are within the control limits and when I have a data point outside the upper control limit I believe I understand that that means this operator is measuring with higher variability (lower repeatability) compared to the rest.

However, the one that I am having a bit more trouble explaining to myself is if an operator's measurements are outside the lower limit. Is that a bad thing? Does this not signify that their overall repeatability is quite good, yet it is still marked red. Does it make sense to have a lower limit or am I missing something in the explanation?

Below is an example of my result. The control limits have different width as some of the group sizes are slightly different due to the nature of the experiment.

Any help is appreciated.

[#2]

Unfortunately I don't see how the data was acquired?  In other words, it is not clear what the rational subgroup is.  In most measurement studies, the operator is between subgroup and therefore cannot possible affect the range (or stand deviation) chart.  You need to understand what components of variation are captured within subgroup.  Those are the only possible explanations for points outside control limits on the range/std dev chart.

Also, you should understand, points outside the control limits on the range/std dev charts are indications of inconsistency/instability due to the sources of variation changing within subgroup.  Points outside control limits have nothing to do with "good/bad."

[#3]

Thanks statman.

I can give more information. There was multiple parts measured on a piece of equipment with each of the 5 operators measuring the same part 6 times on different days. However, some of the days the operators used a slightly different technique to measure and that was recorded. In addition to that, the first 3 days samples were foiled in a specific way which could influence the measurement. After the three days they were unfoiled and measured again and this was all recorded.

The particular plot I showed is for when the data has been filtered when the samples were foiled (so for first 3 measurement days) and measured via measurement technique 1.

Is this the information you were asking about?

[#4]

From your graph, it appears you have an inconsistent number of data points within subgroup? It needs to understood whether the sampling is nested, systematic or crossed. It is always better if you can draw the sampling plan like this:

In any case, what was changing when the standard deviation was calculated?  Is data 3 measures of the same foiled sample by the same operator on the same day using the same technique? If so, then the repeatability precision of the measurement equipment is what is affecting the standard deviation.  While there is insufficient data to draw any conclusion, the data suggests there is evidence the repeatability of the equipment may be unstable.

[#5]

I guess I was mostly trying to get to a conceptual understanding of the meaning of the test-retest plot. The plot is constructed from the standard deviation of different measurements for each part grouped by any other variable used (in this case operator). Normally, we would expect lower standard deviation to be a desirable thing no? Thus, does having a lower control limit in this context make physical sense?

Or conversely, if a point is below the control limit (between operators) that means that one operator has a different measurement spread compared to the others but again lower standard deviation in measurements is a desirable thing right? In this case however, it is marked red when outside the limits whether on the lower or higher side

[#6]

Those are not control limits (CL) but rather decision limits (DL) Analysis of Means Reports, Example of a Detailed Measurement Systems Analysis

Test-Retest Error Comparison Measurement Systems Analysis Report Options

Shows or hides an Analysis of Means for Variances or Analysis of Means Ranges chart for testing if any of the groups have different test-retest error levels. This chart shows if there are differences in the test-retest error between operators. The Analysis of Mean Ranges chart is displayed when ranges are used for variance components.