topic Significance testing on standard deviations in Discussions

Significance testing on standard deviations

Alicia — Mon, 12 Jun 2023 09:38:18 GMT

Hi there,

I have 30 samples measured using two methods (Method A and Method B) and each sample has been measured 5 times. I have summarised my dataset so I have the mean and standard deviation for each sample on the two methods.

I want to compare the within-sample variability between Method A and Method B, matched by sample. Is it ok to perform a paired t-test (or non-parametric equivalent) on the sample standard deviations? Or can this type of analysis only be done on sample means?

Thank you

Re: Significance testing on standard deviations

Victor_G — Mon, 12 Jun 2023 11:57:38 GMT

Hi @Alicia,

For assessing the (un)equality of variances, there are other tests available that could help you compare the methods, in the "Fit Y by X" platform (specifying your response as Y, method as X, and sample ID in the "By" panel for example), in the red triangle under "Unequal Variances" : Unequal Variances Reports (jmp.com)
A paired t-test will look at the differences for each sample ID between method A and B, but it doesn't require nor imply that the two methods have the same variance.

You can also look at the platform "Matched Pairs" available in "Analyze", "Specialized Modeling", "Matched Pairs" : Matched Pairs Platform Options (jmp.com), but this platform is more used for analysis of differences in means, not in variances.

However, as you have a sample/part ID and you want to assess reproducibility of the measurements with two methods, I think the platform "Measurement Systems Analysis (jmp.com)" under "Analyze", "Quality and Process" might also be informative for you. You can specify your sample ID, method (in my example it's "Operator" in X, Grouping), and measurement result :

When the analysis is launched, you can then click on the red triangle from "Measurement Systems Analysis for" (your response), and have a look at the results from AIAG Gauge results R&R or EMP Gauge R&R. This will help you determine the repartition of variance measurements between the method (reproducibility), the part-to-part variation, and the repeatability (since you have 5 repetitions for each sample and method) :

This may not be rigourously what you expect to do, a statistical testing for variances, but it might provide you a more global and informative overview on variance repartition of your measurements between repeatability, reproducibility and part-to-part variation.

I hope this first answer may help you, don't hesitate to provide a toy dataset to better illustrate your needs if I missed the points or if you would like to have more details in the analysis.

Re: Significance testing on standard deviations

MRB3855 — Mon, 12 Jun 2023 12:54:55 GMT

Hi @Alicia , if you have JMP Pro you can do this via the mixed model platform (Sample is a random effect, and Method is a fixed effect). You can define a model where each method has a unique residual (within sample) variance (this phenomena is often called heteroskedasticity). Part of the results will be a comparison of that model to the "null" model (the "usual" constant variance model, often called homoskedasticity).

Re: Significance testing on standard deviations

Alicia — Fri, 16 Jun 2023 06:54:00 GMT

Thank you @Victor_G and @MRB3855 for those great suggestions. I will give them a try