cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • We’re improving the Learn JMP page, and want your feedback! Take the survey
Choose Language Hide Translation Bar
bbenny7
Level III

Help with statistical analysis of repeated measurements

Let's say that I have to compare two equipments and check if they measure the same.

I know that I have some measurment error/uncertainty so I decide to measure the same sample several times at each equipment.

The data collection looks like the table Statistical analysis data collection (see attachment). My idea was to take an average for each sample, and then perform a matched-pairs analysis (see attached Statistical analysis matched-pairs). In example, I can see that the two equipments measure diffently.

Statistical analysis matched-pairs - Matched Pairs of A, B.png

However, if I compare the data in the table Statistical analysis data collection  with Fit Y-by-X and run a non-parametric test for each sample, I can see that the differences are not statistical significant for all the samples.

Statistical analysis data collection - Fit Y by X of value by equipment.png

Waht is the correct way to analyse these data?

1 ACCEPTED SOLUTION

Accepted Solutions
MRB3855
Super User

Re: Help with statistical analysis of repeated measurements

Hi @bbenny7 : I'd consider a mixed model. The Matched pairs analysis (paired t-test) you carried out is a special case of this mixed model.

 

In the Fit Model platform, Value is Y, and the effects are Equipment and Sample. Sample should be considered a random effect (via Attribute menu) so that Sample to sample variation is removed and comparisons are made within samples.

 

And you may want to ask yourself if the difference (statistically significant or not) is meaningful. i.e., the confidence interval may be very tight, and it may not include zero (indicating statistical significant different)...but is that confidence interval a practically meaningful difference? For example, in your matched pairs analysis, the conf int on the difference is -2.41 to -1.15; that is a very small difference.  From a practical/scientific point of view, is that difference (somewhere between -2.41 and -1.15) relevant? 

 

Edit: Also, looking at the plots, it appears the data are rounded to 1 dp; it is not advised to analyze rounded data.  The rounded data may be perhaps what you report, but for data analysis always use the number of decimal places to the precision of the measurement.

 

 

View solution in original post

8 REPLIES 8
Ressel
Level VII

Re: Help with statistical analysis of repeated measurements

Not a statistician, but this looks like an application for variability analysis (Analyze > Quality and Process > Variability / Attribute Gauge Chart) or MSA (Analyze > Quality and Process > Measurement Systems Analysis)

 

Ressel_0-1747752559652.png

 

bbenny7
Level III

Re: Help with statistical analysis of repeated measurements

@Ressel thanks for your answer.

I have tried the Variability/Attribute Gauge Chart, and the result is similar to the matched-pairs.

My concern is that, in this way, I am working with averages, but the limited number of samples and the large spread lead to a large uncertainty in the average estimation.

Is it still fine to work with averages?

Ressel
Level VII

Re: Help with statistical analysis of repeated measurements

@bbenny7 I need to repeat my warning that I'm not a statistician, but looking at your more detailed table Statistical analysis data collection.jmp, I am uncertain what the denomination "Sample" means. Are you expecting actual differences between them, and if so do you expect these differences to be due to 1, 2 and 3 having different properties or are these differences perhaps a result of sampling variation?

Looking at the plot below, as a first approximation, would it be enough to assume samples 1, 2 and 3 are identical (or prove it with via ANOVA) and then compare the group means for equipment A and B against each other?

 

Ressel_0-1747824996947.png

Also, your comment "the large spread lead to a large uncertainty in the average estimation" is already a conclusion that sounds potentially useful for communication to whoever may be interested in the results.

 

AlbertoJD2000
Level II

Re: Help with statistical analysis of repeated measurements

Good point on the ANOVA approach, if samples 1, 2, and 3 aren’t fundamentally different, pooling them could simplify things a lot. I’m also curious what the expected variation between the samples is meant to reflect.

MRB3855
Super User

Re: Help with statistical analysis of repeated measurements

Hi @bbenny7 : I'd consider a mixed model. The Matched pairs analysis (paired t-test) you carried out is a special case of this mixed model.

 

In the Fit Model platform, Value is Y, and the effects are Equipment and Sample. Sample should be considered a random effect (via Attribute menu) so that Sample to sample variation is removed and comparisons are made within samples.

 

And you may want to ask yourself if the difference (statistically significant or not) is meaningful. i.e., the confidence interval may be very tight, and it may not include zero (indicating statistical significant different)...but is that confidence interval a practically meaningful difference? For example, in your matched pairs analysis, the conf int on the difference is -2.41 to -1.15; that is a very small difference.  From a practical/scientific point of view, is that difference (somewhere between -2.41 and -1.15) relevant? 

 

Edit: Also, looking at the plots, it appears the data are rounded to 1 dp; it is not advised to analyze rounded data.  The rounded data may be perhaps what you report, but for data analysis always use the number of decimal places to the precision of the measurement.

 

 

bbenny7
Level III

Re: Help with statistical analysis of repeated measurements

Hi @MRB3855 ,

 

thanks for your answer.

 

I have one more question about the parameter estimates: I cannot see equipment [B] and the intercept doesn't match the value for equipment [B]. How should I calculate the value for equipment [B]?

 

bbenny7_0-1748423252731.png

 

MRB3855
Super User

Re: Help with statistical analysis of repeated measurements

Hi @bbenny7 : It's the way the model is parameterized

-Edit: Details here if interested: https://www.jmp.com/support/help/en/18.0/index.shtml#page/jmp/nominal-factors.shtml#

 

Intercept is interpreted as the average across both pieces of equipment.

A =  intercept + 0.8888889

B =  intercept - 0.8888889.

 

You can see this and comparisons etc if you open the "Effect Details" output.

Xinghua
Level III

回复: Help with statistical analysis of repeated measurements

You should to try "Equivalence Test".

Recommended Articles