topic Is there any way to compare two small datasets ? in Discussions

Is there any way to compare two small datasets ?

ELH — Fri, 23 Feb 2024 09:56:43 GMT

Hello,

I have samples from 3 groups and I want to compare them and check if there are statistical differences.
The problem is that my sample set is very small (group-1: 3 samples, group-2: 2 samples, group-3: 3 samples), so I cannot test for normality and then decide whether to use parametric or non-parametric tests
So what to do?
- I can just use ANOVA followed by Tukey and explain that normality was assumed due to the small sample size.
- Should I use a non-parametric test?

I would appreciate your feedback

Re: Is there any way to compare two small datasets ?

dlehman1 — Fri, 23 Feb 2024 12:07:38 GMT

I personally wouldn't conduct any type of statistical test with these small samples. Graphical exploration would seem appropriate and the results might be suggestive of things to be analyzed further with larger samples. If there are to be no further samples, I still think graphical comparisons would be more valid than any test you might run. Even if you found a test that gives a small p value for some difference between the groups, I wouldn't rely on that for decision making.

Re: Is there any way to compare two small datasets ?

MRB3855 — Mon, 26 Feb 2024 13:23:43 GMT

Hi @ELH : to follow on with what @dlehman1 said: Studies like this can also suffer from being underpowered. i.e., you may be lulled into claiming there is no difference among groups if your p-value isn't sufficiently low. When what could really be going on is that there really is a difference, it's just that with such small sample sizes the probability of showing it (power) is very small.