topic Re: Help with statistical analysis of repeated measurements in Discussions

Help with statistical analysis of repeated measurements

bbenny7 — Tue, 20 May 2025 14:28:02 GMT

Let's say that I have to compare two equipments and check if they measure the same.

I know that I have some measurment error/uncertainty so I decide to measure the same sample several times at each equipment.

The data collection looks like the table Statistical analysis data collection (see attachment). My idea was to take an average for each sample, and then perform a matched-pairs analysis (see attached Statistical analysis matched-pairs). In example, I can see that the two equipments measure diffently.

However, if I compare the data in the table Statistical analysis data collection with Fit Y-by-X and run a non-parametric test for each sample, I can see that the differences are not statistical significant for all the samples.

Waht is the correct way to analyse these data?

Re: Help with statistical analysis of repeated measurements

Ressel — Tue, 20 May 2025 14:51:43 GMT

Not a statistician, but this looks like an application for variability analysis (Analyze > Quality and Process > Variability / Attribute Gauge Chart) or MSA (Analyze > Quality and Process > Measurement Systems Analysis)

Re: Help with statistical analysis of repeated measurements

bbenny7 — Wed, 21 May 2025 08:58:30 GMT

@Ressel thanks for your answer.

I have tried the Variability/Attribute Gauge Chart, and the result is similar to the matched-pairs.

My concern is that, in this way, I am working with averages, but the limited number of samples and the large spread lead to a large uncertainty in the average estimation.

Is it still fine to work with averages?

Re: Help with statistical analysis of repeated measurements

Ressel — Wed, 21 May 2025 11:03:40 GMT

@bbenny7 I need to repeat my warning that I'm not a statistician, but looking at your more detailed table Statistical analysis data collection.jmp, I am uncertain what the denomination "Sample" means. Are you expecting actual differences between them, and if so do you expect these differences to be due to 1, 2 and 3 having different properties or are these differences perhaps a result of sampling variation?

Looking at the plot below, as a first approximation, would it be enough to assume samples 1, 2 and 3 are identical (or prove it with via ANOVA) and then compare the group means for equipment A and B against each other?

Also, your comment "the large spread lead to a large uncertainty in the average estimation" is already a conclusion that sounds potentially useful for communication to whoever may be interested in the results.

Re: Help with statistical analysis of repeated measurements

AlbertoJD2000 — Tue, 27 May 2025 06:47:03 GMT

Good point on the ANOVA approach, if samples 1, 2, and 3 aren’t fundamentally different, pooling them could simplify things a lot. I’m also curious what the expected variation between the samples is meant to reflect.

Re: Help with statistical analysis of repeated measurements

MRB3855 — Tue, 27 May 2025 10:03:14 GMT

Hi @bbenny7 : I'd consider a mixed model. The Matched pairs analysis (paired t-test) you carried out is a special case of this mixed model.

In the Fit Model platform, Value is Y, and the effects are Equipment and Sample. Sample should be considered a random effect (via Attribute menu) so that Sample to sample variation is removed and comparisons are made within samples.

And you may want to ask yourself if the difference (statistically significant or not) is meaningful. i.e., the confidence interval may be very tight, and it may not include zero (indicating statistical significant different)...but is that confidence interval a practically meaningful difference? For example, in your matched pairs analysis, the conf int on the difference is -2.41 to -1.15; that is a very small difference. From a practical/scientific point of view, is that difference (somewhere between -2.41 and -1.15) relevant?

Edit: Also, looking at the plots, it appears the data are rounded to 1 dp; it is not advised to analyze rounded data. The rounded data may be perhaps what you report, but for data analysis always use the number of decimal places to the precision of the measurement.

Re: Help with statistical analysis of repeated measurements

bbenny7 — Wed, 28 May 2025 09:11:17 GMT

Hi @MRB3855 ,

thanks for your answer.

I have one more question about the parameter estimates: I cannot see equipment [B] and the intercept doesn't match the value for equipment [B]. How should I calculate the value for equipment [B]?

Re: Help with statistical analysis of repeated measurements

MRB3855 — Wed, 28 May 2025 14:13:09 GMT

Hi @bbenny7 : It's the way the model is parameterized.

-Edit: Details here if interested: https://www.jmp.com/support/help/en/18.0/index.shtml#page/jmp/nominal-factors.shtml#

Intercept is interpreted as the average across both pieces of equipment.

A = intercept + 0.8888889

B = intercept - 0.8888889.

You can see this and comparisons etc if you open the "Effect Details" output.

回复： Help with statistical analysis of repeated measurements

Xinghua — Tue, 03 Jun 2025 05:04:47 GMT

You should to try "Equivalence Test".