Share your ideas for the JMP Scripting Unsession at Discovery Summit by September 17th. We hope to see you there!
Choose Language Hide Translation Bar
Highlighted
FR60
Level III

two way anova with strongly unbalanced samples

In case like this, where the two sample sizes are so strongly different (25 vs 410), the standard anova test is it ok?  

 

Picture1.png

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: two way anova with strongly unbalanced samples

Use the Unequal Variance command in the Oneway platform menu. It will not only provide several hypothesis tests for the variance but also the Walsh ANOVA for the cases with unequal variances.

Learn it once, use it forever!

View solution in original post

6 REPLIES 6
Highlighted
statman
Level VII

Re: two way anova with strongly unbalanced samples

I assume you mean the F-test?  Before doing any quantitative statistical test I would examine the data for unusual data points (special or assignable).  Understand these first before doing any quantitative test.  It looks to me as the means are about the same, but the variances of the 2 groups are different.  I have no context for the scale, so I don't know if the variation in those values is of any practical significance.  The MSE will be estimated by the average of the variances of the 2 groups.  In this case, that variance will likely be large, so the test of differences will likely yield no significant difference between the mean of the 2 groups.

Highlighted
statman
Level VII

Re: two way anova with strongly unbalanced samples

Just to add, whether to believe those results or whether those results can be extrapolated is largely a function of how the data was acquired and how representative those data sets are of the future.
Highlighted
FR60
Level III

Re: two way anova with strongly unbalanced samples

I means the pval. In this case it is not significant with 95% CL. But I can believe it? I saw other cases like this with strong unbalanced sample with a pval lower than 0.01. You are rogth. The "Yes" group has a large variance than "No " group. So in this case which the best method to compare the 2 groups? Big difference in variance and in sample sizes.

Felice
Highlighted

Re: two way anova with strongly unbalanced samples

Use the Unequal Variance command in the Oneway platform menu. It will not only provide several hypothesis tests for the variance but also the Walsh ANOVA for the cases with unequal variances.

Learn it once, use it forever!

View solution in original post

Highlighted

Re: two way anova with strongly unbalanced samples

The difference in the sample size of the two groups does not matter, except that it inflates the variance of the parameter estimates. (Yes, there is an ANOVA model.)

 

You need to check the assumptions of the ANOVA to decide if the inference is valid.

 

  • Observations are independent (no auto-correlation).
  • Errors are normally distributed.
  • Variance is constant across groups. The only difference is the mean.

Looks like you severely violate at least two of the assumptions, so you should not use ANOVA.

Learn it once, use it forever!
Highlighted
FR60
Level III

Re: two way anova with strongly unbalanced samples

Thanks Mark. So in that case which is the best model to compare the 2 groups?

F.