Learn more in our free online course:
Statistical Thinking for Industrial Problem Solving
In this video, you learn how to compare more than two means using one-way analysis of variance, or ANOVA. For this video, we use the file Michelson 1879.jmp in the course data. In this scenario, we test the null hypothesis that the mean velocity is equal for all five trials against the alternative hypothesis that at least two means are different.
To do this, we select Fit Y by X from the Analyze menu. We select Velocity as the Y, Response and Trial # as the X, Factor.
Then we select Means/Anova from the red triangle menu.
JMP produces mean diamonds, to help interpret the test results. You can see that not all of the diamonds overlap. For example, the first diamond doesn’t overlap with the diamonds for trials 4 and 5. This indicates that at least some of the means are different from one another.
For a more formal analysis, we look at the Analysis of Variance table.
The p-value for this test, reported as Prob > F, is 0.0031. We can reject the null hypothesis that all of the means are equal.
To determine which means are different, we can use a multiple comparison procedure.
Several procedures are available in JMP, under the red triangle menu and then Compare Means.
We’ll select the first option, Each Pair, Student’s t. This enables you to do all possible two-sample t tests.
The statistical output at the bottom indicates that four pairs of means are different at a significance level of 0.05. For example, the p-value for the test comparing the mean of the first trial to the mean of the fourth trial is 0.0003, which is highly significant.
Let’s look at the graph. The circles are comparison circles. You can click on a circle to conduct a two-sample t-test between the corresponding mean and all other means.
The top circle corresponds to the mean for the first trial. When you click on this circle, it turns red, and the label on the X axis also turns red. Notice that all of the other circles and labels are gray. This tells you that the mean for Trial 1 is different from all of the other means.
When we click on any other circle, notice that the only circle that is gray is for the mean of Trial 1. This tells us that the mean for Trial 1 is different from all of the other means and that none of the other means are different from one another.
However, with this test, we can conduct 10 two-sample t tests. To protect us from finding a false positive, we should use another procedure.
When we select All Pairs, Tukey HSD (for Honestly Significant Difference), the overall error rate, across all of the comparisons, is 0.05, or whatever value you select.
Notice that the circles are larger. This indicates that the test is less sensitive than the Each Pair, Student’s t tests. As a result, Tukey’s finds fewer differences.
Let’s talk about the other tests. You would use With Best, Hsu MCB (for Multiple Comparison with Best) when you are interested in only making comparisons to what you specify as the "best" mean. This is usually the highest or the lowest.
And you would use With Control, Dunnett’s when you are testing means against the mean of a standard or control group.
Now, what if your question is: "Which of these means is different from the overall mean?" To answer this question, you can use analysis of means, or ANOM.
To do this, we select Analysis of Means from the top red triangle. There are several procedures available. We’ll select the first option, ANOM.
From this analysis, we see that only the mean for the first trial is significantly different from the overall mean.
As you can see, there are many methods available for comparing means, either with one another or with the overall mean. The procedure you choose depends largely on the question that you are asking.
