topic Re: A Cringeworthy Statistic Statement: #3 in a series in Discussions

A Cringeworthy Statistic Statement: #3 in a series

JerryFish — Fri, 09 Jun 2023 00:42:02 GMT

Welcome back for the third Cringeworthy discussion topic! Many thanks to those that contributed to the first two topics. Please feel free to join this discussion as well!

Today's Cringeworthy Statement should be obvious, though you all have surprised me with great insights in the past that I hadn't considered! Here it is:

An engineer is tasked with running a test on a particular process output. The goal is to learn about the average performance of the process. The engineer decides to measure the process output three times.

After completing the test, the engineer reports the following in a stakeholder meeting:

"The process output is 11.327384, based on averaging 3 independent measurements."

What is cringeworthy about this statement?

#cringeworthy #measurements #msa

Re: A Cringeworthy Statistic Statement: #3 in a series

Ressel — Tue, 16 Nov 2021 15:22:56 GMT

For starters:

Unit of measurement is not given
How are the measurements independent?
Measure of dispersion is missing. Perhaps reproducibility / repeatability does really only have an effect 5-6 digits after the comma, but one could easily and would intuitively doubt a result like this.

Re: A Cringeworthy Statistic Statement: #3 in a series

dale_lehman — Tue, 16 Nov 2021 16:37:17 GMT

If the measurement instrument is accurate enough to report the result to 6 decimal places, then it is surely enough to provide the degree to which the 3 measurements differed. So, yes, I want to see some measure of dispersion at a minimum. Then, of course, all the additional caveats (how do we know they are independent? how are the measurements taken? were the conditions the same? etc.).

Re: A Cringeworthy Statistic Statement: #3 in a series

Craige_Hales — Tue, 16 Nov 2021 16:53:48 GMT

Units! Somebody came back from a vacation in Germany several years ago. I asked about filling up the tank in the rental car. "Gas was about half as expensive in Germany as the USA." Comparing Germany's 1.653 EUR/Litre vs USA's 3.763 USD/Gallon, I guess it did seem like that.

(No time to do the math? The reverse is true. At least in late 2021. Your mileage may vary.)

Re: A Cringeworthy Statistic Statement: #3 in a series

statman — Tue, 16 Nov 2021 17:02:48 GMT

More fun...there are many issues with the statement.

Head in the oven, feet in the freezer, on average OK...

An average without the context of variation is a worthless statistic. Not even sure if it is the appropriate measure of central tendency.

How does reporting this statistic have anything to do with the "goal". There is no description of over what changing factors in the process is this measure indicative of. There is no sampling plan provided (or description of the process associated with when the 3 measures were taken).

There is no estimate of measurement precision. Just because there are many decimal places reported means nothing (it could simply be a reporting issue, we don't know the measurement unit). we have no knowledge of the accuracy, precision, discrimination or stability of the measurement process.

Not sure what you can do with 3 measurements unless they possibly indicate an issue so as to motivate you to investigate.

Re: A Cringeworthy Statistic Statement: #3 in a series

P_Bartell — Wed, 17 Nov 2021 12:13:10 GMT

"...3 independent measurements." Something that isn't clear is sample size wrt to the process. Is it 1? Is it 3? Not stated.

Re: A Cringeworthy Statistic Statement: #3 in a series

worldwyn — Tue, 23 Nov 2021 18:43:06 GMT

What is the process output measuring, what is the target, and what are the units? What are the conditions, e.g. dates, shifts, and other relevant conditions? What are the sample sizes for the three measurements? Can each sample mean be considered to be different from the others or not (cannot reject the null hypothesis in Student t test or do the 95% confidence intervals for the sample means overlap?). Only after "cannot be considered different" can you justifiably pool the samples and calculate a grand mean (with an appropriate number of decimal places). Then should included the 95% confidence interval in the statement to stakeholders so they understand the significance of the statement.

Re: A Cringeworthy Statistic Statement: #3 in a series

Craige_Hales — Tue, 23 Nov 2021 22:32:08 GMT

@JerryFish I'm hoping the blog covers this independent thing. Right now, I'm thinking it means three separate rulers, but I'm not sure why that would be a good thing.

Re: A Cringeworthy Statistic Statement: #3 in a series

Georg — Thu, 02 Dec 2021 14:43:52 GMT

As others pointed out, there are many details missing. Without the unit, boundary conditions and other details the statement is simply useless. It also would be an important information before doing the thest for the output, what the number is needed for.

Re: A Cringeworthy Statistic Statement: #3 in a series

PatrickGiuliano — Fri, 03 Dec 2021 03:35:02 GMT

not much you can say with a confidence interval based on n of 3. A very wide interval indeed! Same thing if we were making an inference on a specific proportion of the population ("Reliability") at a particular level of Confidence (e.g. 95%). "Law of small numbers". The smaller the sample size the lower the precision we have on the estimate of a sample statistic.

Also, the confidence level we choose should be based on risk which should be based on our practical assessment of the problem and the potential consequences of failure to meet a given criterion or set of criteria.

Re: A Cringeworthy Statistic Statement: #3 in a series

PatrickGiuliano — Fri, 03 Dec 2021 03:38:13 GMT

Would be better if the engineer plotted the data in Graph Builder. That way they would have been forced to show the "spread" (visually) and likely would have defined the units (since their axes would need to be labeled for the graph to make any sense). The exercise would have probably have triggered them to put some limit(s) on the graph that are of practical importance, such as process control limits or specification limits. Go Graph Builder!

Re: A Cringeworthy Statistic Statement: #3 in a series

PoissonHyena301 — Tue, 01 Mar 2022 19:12:09 GMT

Hello
As a beginner i will ask:
- what units are define
- how samples were taken : 3 one after one or 3 with some break time beetween
- why 3?
- what is assumed power of the test?
- what are the CI ?
- what is std of the sample.?
- is this response important to the process
- was the measurment system defined as accurate (GRR)?
- what is a distribution of data?
This for start;)
Pawel