cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Check out the JMP® Marketplace featured Capability Explorer add-in
Choose Language Hide Translation Bar
ELH
ELH
Level III

Is there any way to compare two small datasets ?

Hello,

 

I have samples from 3 groups and I want to compare them and check if there are statistical differences.
The problem is that my sample set is very small (group-1: 3 samples, group-2: 2 samples, group-3: 3 samples), so I cannot test for normality and then decide whether to use parametric or non-parametric tests
So what to do?
- I can just use ANOVA followed by Tukey and explain that normality was assumed due to the small sample size.
- Should I use a non-parametric test?

 

I would appreciate your feedback

2 REPLIES 2
dlehman1
Level V

Re: Is there any way to compare two small datasets ?

I personally wouldn't conduct any type of statistical test with these small samples.  Graphical exploration would seem appropriate and the results might be suggestive of things to be analyzed further with larger samples.  If there are to be no further samples, I still think graphical comparisons would be more valid than any test you might run.  Even if you found a test that gives a small p value for some difference between the groups, I wouldn't rely on that for decision making.

MRB3855
Super User

Re: Is there any way to compare two small datasets ?

Hi @ELH : to follow on with what @dlehman1 said: Studies like this can also suffer from being underpowered. i.e., you may be lulled into claiming there is no difference among groups if your p-value isn't sufficiently low.  When what could really be going on is that there really is a difference, it's just that with such small sample sizes the probability of showing it (power) is very small.