cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
Alicia
Level III

Significance testing on standard deviations

Hi there,

 

I have 30 samples measured using two methods (Method A and Method B) and each sample has been measured 5 times. I have summarised my dataset so I have the mean and standard deviation for each sample on the two methods.

 

I want to compare the within-sample variability between Method A and Method B, matched by sample. Is it ok to perform a paired t-test (or non-parametric equivalent) on the sample standard deviations? Or can this type of analysis only be done on sample means?

 

Thank you

3 REPLIES 3
Victor_G
Super User

Re: Significance testing on standard deviations

Hi @Alicia,

 

For assessing the (un)equality of variances, there are other tests available that could help you compare the methods, in the "Fit Y by X" platform (specifying your response as Y, method as X, and sample ID in the "By" panel for example), in the red triangle under "Unequal Variances" : Unequal Variances Reports (jmp.com)
A paired t-test will look at the differences for each sample ID between method A and B, but it doesn't require nor imply that the two methods have the same variance.

 

You can also look at the platform "Matched Pairs" available in "Analyze", "Specialized Modeling", "Matched Pairs" : Matched Pairs Platform Options (jmp.com), but this platform is more used for analysis of differences in means, not in variances.

 

However, as you have a sample/part ID and you want to assess reproducibility of the measurements with two methods, I think the platform "Measurement Systems Analysis (jmp.com)" under "Analyze", "Quality and Process" might also be informative for you. You can specify your sample ID, method (in my example it's "Operator" in X, Grouping), and measurement result :

Victor_G_0-1686569650830.png

When the analysis is launched, you can then click on the red triangle from "Measurement Systems Analysis for" (your response), and have a look at the results from AIAG Gauge results R&R or EMP Gauge R&R. This will help you determine the repartition of variance measurements between the method (reproducibility), the part-to-part variation, and the repeatability (since you have 5 repetitions for each sample and method) :

 

Victor_G_1-1686569937365.png

This may not be rigourously what you expect to do, a statistical testing for variances, but it might provide you a more global and informative overview on variance repartition of your measurements between repeatability, reproducibility and part-to-part variation.

I hope this first answer may help you, don't hesitate to provide a toy dataset to better illustrate your needs if I missed the points or if you would like to have more details in the analysis.

 

Victor GUILLER

"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)
MRB3855
Super User

Re: Significance testing on standard deviations

Hi @Alicia , if you have JMP Pro you can do this via the mixed model platform (Sample is a random effect, and Method is a fixed effect). You can define a model where each method has a unique residual (within sample) variance (this phenomena is often called heteroskedasticity). Part of the results will be a comparison of that model to the "null" model (the "usual" constant variance model, often called homoskedasticity).

Alicia
Level III

Re: Significance testing on standard deviations

Thank you @Victor_G  and @MRB3855 for those great suggestions. I will give them a try