cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
FR60
Level IV

two way anova with strongly unbalanced samples

In case like this, where the two sample sizes are so strongly different (25 vs 410), the standard anova test is it ok?  

 

Picture1.png

1 ACCEPTED SOLUTION

Accepted Solutions

Re: two way anova with strongly unbalanced samples

Use the Unequal Variance command in the Oneway platform menu. It will not only provide several hypothesis tests for the variance but also the Walsh ANOVA for the cases with unequal variances.

View solution in original post

6 REPLIES 6
statman
Super User

Re: two way anova with strongly unbalanced samples

I assume you mean the F-test?  Before doing any quantitative statistical test I would examine the data for unusual data points (special or assignable).  Understand these first before doing any quantitative test.  It looks to me as the means are about the same, but the variances of the 2 groups are different.  I have no context for the scale, so I don't know if the variation in those values is of any practical significance.  The MSE will be estimated by the average of the variances of the 2 groups.  In this case, that variance will likely be large, so the test of differences will likely yield no significant difference between the mean of the 2 groups.

"All models are wrong, some are useful" G.E.P. Box
statman
Super User

Re: two way anova with strongly unbalanced samples

Just to add, whether to believe those results or whether those results can be extrapolated is largely a function of how the data was acquired and how representative those data sets are of the future.
"All models are wrong, some are useful" G.E.P. Box
FR60
Level IV

Re: two way anova with strongly unbalanced samples

I means the pval. In this case it is not significant with 95% CL. But I can believe it? I saw other cases like this with strong unbalanced sample with a pval lower than 0.01. You are rogth. The "Yes" group has a large variance than "No " group. So in this case which the best method to compare the 2 groups? Big difference in variance and in sample sizes.

Felice

Re: two way anova with strongly unbalanced samples

Use the Unequal Variance command in the Oneway platform menu. It will not only provide several hypothesis tests for the variance but also the Walsh ANOVA for the cases with unequal variances.

Re: two way anova with strongly unbalanced samples

The difference in the sample size of the two groups does not matter, except that it inflates the variance of the parameter estimates. (Yes, there is an ANOVA model.)

 

You need to check the assumptions of the ANOVA to decide if the inference is valid.

 

  • Observations are independent (no auto-correlation).
  • Errors are normally distributed.
  • Variance is constant across groups. The only difference is the mean.

Looks like you severely violate at least two of the assumptions, so you should not use ANOVA.

FR60
Level IV

Re: two way anova with strongly unbalanced samples

Thanks Mark. So in that case which is the best model to compare the 2 groups?

F.