turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Discussions
- :
- One-tail vs. Two-tail tests

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Mar 27, 2009 4:17 PM
(2694 views)

I am doing the means comparison using one-way ANOVA in Jmp. When I look at the detailed report, I see the p-values for both one-tail and two-tail t-tests.

However, the comparison circles seem to display according only to the two-tail confidence levels. The same about LSD (least significant difference) - it is calculated only for the two-tailed test.

Question: can I change it somewhere so that one-tail test will be used as default? If not, can I get the circles and lsd values for one-tail test somehow?

Thanks,

Mike.

10 REPLIES

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Mar 31, 2009 7:15 AM
(2216 views)

If you want to do the one-sided test to see if the mean of group 1 is less than the mean of group 2, you go to comparison circles, and if they indicate a difference and the mean of group 1 is less than group 2 ... you have found a difference. If you go to the comparison circles and they indicate a difference but the mean of group 1 is greater than the mean of group 2, you have found no significant difference. You might have to adjust the alpha level of the test to get exactly what you want.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Mar 31, 2009 11:10 AM
(2216 views)

I am certainly ready to change whatever is needed to get the right result. But first, let me clarify my point.

The radii of the circles are T(alpha)*Sigma, where

T(alpha) is an alpha-quantile of the Student t-distribution,

Sigma is the group standard deviation.

Alpha is the type I error that you suggest to change if I want. I understand this. Now, suppose I have default Alpha = 0.5 and I performed the t-test on my 2 data groups and see that "Prob > t" is equal to 0.49.

If I know in advance which group is higher, and I am not interested in the situation when this group is significantly lower, I operate in one-tail test realm, and the "Prob >t" is exactly what I need. So, if this value is less than 0.5, then my groups are significantly different.

But if I click on the circles, they show no significant difference. This is because both the LSD (least significant difference) and the circles radii are calculated from the two-tailed error level equal 2*Alpha.

In my example 2*Alpha = 0.98 > 0.5

The 2-tail approach makes sense in certain situations, but my question is - can I change it to one-tail? It so happened, I do know in advance that one dataset can be only higher, and I want to test ONLY this for my error level.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 5:19 AM
(2216 views)

The default in JMP is to do a two-sided test, so when alpha = 0.5, there is 0.25 in each tail. So, when you set alpha = 0.5 in JMP, the one-sided test as I described it, will be at alpha=0.25. That doesn't sound like what you want.

You want to do a one-sided test with alpha = 0.5, and that is impossible to do in JMP. You would need to set alpha = 1, and I'm pretty sure JMP will give you an error if you try it.

Pencil and paper, you could do the math and come up with a one-sided test at alpha=0.5. JMP doesn't allow you to do this for alpha=0.5, but does allow you to do this for any alpha less than 0.5, using the method I described.

Performing hypothesis tests at alpha = 0.5 is not only unusual, but seems to me to be a crazy idea. Flipping coins will give you the same results. I recommend you not perform hypothesis tests at alpha = 0.5.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 10:18 AM
(2216 views)

I am sorry. Of course it's a typo.

Alpha = 0.05, it's a default value.

The test shows " Prob > t = 0.049", meaning 1-tail difference is significant.

Circles show the difference is unsignificant, because it uses 2-tali test numbers, and 2*Alpha = 0.098 > 0.05

LSD table title reads "ABS(Dif) - LSD" suggesting also 2-tail test (absolute value of difference).

You said JMP can't do one-tailed test circles - I got it, thanks for the clarification.

Now, this your statement:

>>The default in JMP is to do a two-sided test, so when alpha = 0.5, there is 0.25 in each tail.

is wrong. It looks like JMP is really doing two-sided test, but if I set Alpha = 0.05, there is 0.05 in EACH tail, so the real error level for the mean difference is 0.1 !

This is the second part of my question - is this is a bug or is it by design?

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 11:10 AM
(2216 views)

> >>The default in JMP is to do a two-sided test, so

> when alpha = 0.5, there is 0.25 in each tail.

>

> is wrong.

> It looks like JMP is really doing two-sided

> test, but if I set Alpha = 0.05, there is 0.05 in

> EACH tail, so the real error level for the mean

> difference is 0.1 !

Really? How do you know this?

I ask because when I look in JMP Help, in a chapter entitled "Details of Comparison Circles", it sure looks to me like they are claiming that the comparison circles are based upon alpha/2 in each tail.

> This is the second part of my question - is this is a

> bug or is it by design?

Since I don't work for JMP, I cannot answer this.

Message was edited by: Paige

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 11:28 AM
(2216 views)

1. Take any 2 datasets with reasonably different means - or take any dataset and split it in two parts.

2. Run "one-way ANOVA" t-test, and note the value of "Prob > t". Lets suppose it is equal to X.

3. Go to "Set Alpha level" -> "Other" and set Alpha equal to the value X+0.01

4. Go to "Compare means" -> "Each pair, Student's t" and click on the circles to see that they are claimed not significantly different.

5. Change Alpha level to 2*X-0.01 and repeat (4) with the same result

6. Change Alpha level to 2*X+0.01 and repeat (4) to see that the difference becomes significant.

7. You may also see this on the t-test report in the value "Prob > Abs(t)" - it is equal to 2*X, and this seems to be used in the significance test.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 11:44 AM
(2216 views)

I created a very small data set, so I can do the calculations by hand. The prob > |t| is very clearly the two-tailed probability.

If you look at the comparison circles in my specific data set, the means are not different at alpha = 0.05. Matches my hand calculations. For this data set, if you set the alpha to exactly equal the prob > |t|, the circles still show no difference. If you increase the alpha to be slightly greater than the prob > |t|, the circles now show a statistically significant difference. This is exactly what you would expect.

So, the comparison circles use two-tailed tests.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 11:54 AM
(2216 views)

Problem is different: if you set the error level = Alpha in Jmp GUI, then Jmp uses two-tail test with error level 2*Alpha!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Apr 1, 2009 1:01 PM
(2216 views)

I do not agree that "Problem is different: if you set the error level = Alpha in Jmp GUI, then Jmp uses two-tail test with error level 2*Alpha!"

I say ... JMP uses two-tail test with error level alpha.