Subscribe Bookmark RSS Feed

How do you interpret Welch's Test results?

jeff_kolton1

Community Trekker

Joined:

Jun 25, 2014

Since the tests for Unequal Variances indicates that the variances ARE unequal, how do I interpret the Welch's Test results?  My take, based on the JMP Help is that since the Prob > F is greater than 0.05, the means of my two sample groups are NOT significantly different.

10073_pastedImage_0.png

1 ACCEPTED SOLUTION

Accepted Solutions
mdawson69

Community Trekker

Joined:

Aug 26, 2015

Solution

jeff.kolton1 wrote:

My take, based on the JMP Help is that since the Prob > F is greater than 0.05, the means of my two sample groups are NOT significantly different.

That is true given that you are testing at the 5% significance level. When using p-values to determine the results of a test the general rule is no significant differences were detected if p > α; in JMP α = 0.05 by default. JMP also shows you significance by color-coding the p-values. If you look at the results of your tests for equal variances, you can see that all of the tests are significant at the 5% level. Note that the Levene, Bartlett, and 2-sided F Test are also significant at the 1% level. My guess, is that the color coding, which I think started with JMP 12, distinguishes between the 5% level (red) and 1% level (orange).

I am not aware of any global way to change the default significance level, but I know I have seen and option to select the significance level for tests in specific platforms.

3 REPLIES
mdawson69

Community Trekker

Joined:

Aug 26, 2015

Solution

jeff.kolton1 wrote:

My take, based on the JMP Help is that since the Prob > F is greater than 0.05, the means of my two sample groups are NOT significantly different.

That is true given that you are testing at the 5% significance level. When using p-values to determine the results of a test the general rule is no significant differences were detected if p > α; in JMP α = 0.05 by default. JMP also shows you significance by color-coding the p-values. If you look at the results of your tests for equal variances, you can see that all of the tests are significant at the 5% level. Note that the Levene, Bartlett, and 2-sided F Test are also significant at the 1% level. My guess, is that the color coding, which I think started with JMP 12, distinguishes between the 5% level (red) and 1% level (orange).

I am not aware of any global way to change the default significance level, but I know I have seen and option to select the significance level for tests in specific platforms.

Dan_Obermiller

Joined:

Apr 3, 2013

You can control the formatting that JMP uses for the p-values. Go to File > Preferences. Click the Reports category. Click the "Manage Rules" button next to Show Conditional Formatting.

10082_pastedImage_0.png

From there choose PValue and edit. You then get a dialog box that allows you to change when JMP flags a low p-value.10083_pastedImage_2.png

Note that by using p-values, you never really have to specify your confidence level. You just always use the generic rule: if p-value is less than alpha, reject the null hypothesis. It is the formatting in JMP that may cause issues.

Dan Obermiller
mdawson69

Community Trekker

Joined:

Aug 26, 2015

Good to know. I generally follow the generic rule, but I also have yet to work on anything that required better than the 5% significance level.