For one way ANOVA test, why results from "Fit Y by X" and "Fit Model" are different?

[#1]

Hi,

I was trying to do one-way ANOVA test using "Fit Y by X" and "Fit Least Squares", the sample data that I used is "Snapdragon". Originally, I thought the P value and F Ratio I got from both methods should be the same (see highlighted part). However I was wrong. They are actually quite different

Could someone tell me why they are different using these two methods? Which one is a more accurate analysis?

Many thanks!

[#2 · Accepted solution]

Hi,
The simple answer is the with Fit Y by X you are fitting 2 separate models. With Fit Model you are fitting one model with both effects.
F tests compare the model variance against error variance. The combination of both Soil and Block effects account for more of the variation in Y than either one alone. Therefore error variance is lower and the F ratio is higher.
Does that help?
Regards,
Phil

[#3]

In fact, in this case you can see that the Model Sum of Squares for the combined model (142) is simply the sum of Sum of Squares for the effects in the individual models (103 + 39). Have a look at the in the ANOVA tables - hopefully it makes sense.

(In this case Soil and Block effects are orthogonal - not correlated with each other - because it is data from a designed experiment. In other cases - with observational data - this will not necessarily be the case)

[#4 · Accepted solution]

You also ask, which is the most accurate analysis? Well, all models are wrong...

However, the individual one way ananlyses are comparing the effect variance against error variance. But the error variance is overestimated because it includes variance that can be explained by the missing effect.

Therefore I would suggest that the combined model is the more useful model because the error term is only variance left over after we have accounted for the effects of the factor effects.

It really goes back to the hypothesis that was framed at the beginning of the experiment.

[#5]

Hi Phil,

Thank you for the thorough explanation. That makes more sense now. So if I have a more complicated set of data (several factors, and each factor has >5 levels) and I try to find the most dominant factors, I should do a Fit Model analysis and get conclusion from that result, instead of just looking at the one-way ANOVA result and find the one with the highest F ratio. Is my understanding correct? 

Thanks,
Sichao

[#6]

Yes. If I have several factors that could be affecting the response, I would explore the data in a number of ways and would most likely end with a multivariate model (Fit Model) for my final conclusions. In some cases you may also need to consider interactions between factors and stepwise fitting.

Unfortunately I can't tell you what you would need to do in every situation in the space of a discussion thread. You might want to consider getting some formal training.

[#7]

Got it. This is already very helpful. Thanks!