topic Re: How do you interpret Welch's Test results? in Discussions

How do you interpret Welch's Test results?

jeff_kolton1 — Wed, 19 Oct 2016 00:44:44 GMT

Since the tests for Unequal Variances indicates that the variances ARE unequal, how do I interpret the Welch's Test results? My take, based on the JMP Help is that since the Prob > F is greater than 0.05, the means of my two sample groups are NOT significantly different.

Re: How do you interpret Welch's Test results?

mdawson69 — Wed, 30 Sep 2015 15:18:56 GMT

jeff.kolton1 wrote:

My take, based on the JMP Help is that since the Prob > F is greater than 0.05, the means of my two sample groups are NOT significantly different.

That is true given that you are testing at the 5% significance level. When using p-values to determine the results of a test the general rule is no significant differences were detected if p > α; in JMP α = 0.05 by default. JMP also shows you significance by color-coding the p-values. If you look at the results of your tests for equal variances, you can see that all of the tests are significant at the 5% level. Note that the Levene, Bartlett, and 2-sided F Test are also significant at the 1% level. My guess, is that the color coding, which I think started with JMP 12, distinguishes between the 5% level (red) and 1% level (orange).

I am not aware of any global way to change the default significance level, but I know I have seen and option to select the significance level for tests in specific platforms.

Re: How do you interpret Welch's Test results?

Dan_Obermiller — Wed, 19 Oct 2016 00:45:45 GMT

You can control the formatting that JMP uses for the p-values. Go to File > Preferences. Click the Reports category. Click the "Manage Rules" button next to Show Conditional Formatting.

From there choose PValue and edit. You then get a dialog box that allows you to change when JMP flags a low p-value.

Note that by using p-values, you never really have to specify your confidence level. You just always use the generic rule: if p-value is less than alpha, reject the null hypothesis. It is the formatting in JMP that may cause issues.

Re: How do you interpret Welch's Test results?

mdawson69 — Wed, 30 Sep 2015 18:45:53 GMT

Good to know. I generally follow the generic rule, but I also have yet to work on anything that required better than the 5% significance level.