I am doing the means comparison using one-way ANOVA in Jmp. When I look at the detailed report, I see the p-values for both one-tail and two-tail t-tests. However, the comparison circles seem to display according only to the two-tail confidence levels. The same about LSD (least significant difference) - it is calculated only for the two-tailed test.
Question: can I change it somewhere so that one-tail test will be used as default? If not, can I get the circles and lsd values for one-tail test somehow?
Yes, you have to make a change in your mental process, not in JMP.
If you want to do the one-sided test to see if the mean of group 1 is less than the mean of group 2, you go to comparison circles, and if they indicate a difference and the mean of group 1 is less than group 2 ... you have found a difference. If you go to the comparison circles and they indicate a difference but the mean of group 1 is greater than the mean of group 2, you have found no significant difference. You might have to adjust the alpha level of the test to get exactly what you want.
I am certainly ready to change whatever is needed to get the right result. But first, let me clarify my point.
The radii of the circles are T(alpha)*Sigma, where T(alpha) is an alpha-quantile of the Student t-distribution, Sigma is the group standard deviation.
Alpha is the type I error that you suggest to change if I want. I understand this. Now, suppose I have default Alpha = 0.5 and I performed the t-test on my 2 data groups and see that "Prob > t" is equal to 0.49.
If I know in advance which group is higher, and I am not interested in the situation when this group is significantly lower, I operate in one-tail test realm, and the "Prob >t" is exactly what I need. So, if this value is less than 0.5, then my groups are significantly different.
But if I click on the circles, they show no significant difference. This is because both the LSD (least significant difference) and the circles radii are calculated from the two-tailed error level equal 2*Alpha.
In my example 2*Alpha = 0.98 > 0.5
The 2-tail approach makes sense in certain situations, but my question is - can I change it to one-tail? It so happened, I do know in advance that one dataset can be only higher, and I want to test ONLY this for my error level.
Since you have specified alpha=0.5 in several places, I assume this isn't a typo, but it sure is an unusual way to do things.
The default in JMP is to do a two-sided test, so when alpha = 0.5, there is 0.25 in each tail. So, when you set alpha = 0.5 in JMP, the one-sided test as I described it, will be at alpha=0.25. That doesn't sound like what you want.
You want to do a one-sided test with alpha = 0.5, and that is impossible to do in JMP. You would need to set alpha = 1, and I'm pretty sure JMP will give you an error if you try it.
Pencil and paper, you could do the math and come up with a one-sided test at alpha=0.5. JMP doesn't allow you to do this for alpha=0.5, but does allow you to do this for any alpha less than 0.5, using the method I described.
Performing hypothesis tests at alpha = 0.5 is not only unusual, but seems to me to be a crazy idea. Flipping coins will give you the same results. I recommend you not perform hypothesis tests at alpha = 0.5.
Alpha = 0.05, it's a default value. The test shows " Prob > t = 0.049", meaning 1-tail difference is significant. Circles show the difference is unsignificant, because it uses 2-tali test numbers, and 2*Alpha = 0.098 > 0.05 LSD table title reads "ABS(Dif) - LSD" suggesting also 2-tail test (absolute value of difference).
You said JMP can't do one-tailed test circles - I got it, thanks for the clarification.
Now, this your statement: >>The default in JMP is to do a two-sided test, so when alpha = 0.5, there is 0.25 in each tail.
is wrong. It looks like JMP is really doing two-sided test, but if I set Alpha = 0.05, there is 0.05 in EACH tail, so the real error level for the mean difference is 0.1 !
This is the second part of my question - is this is a bug or is it by design?
> Now, this your statement: > >>The default in JMP is to do a two-sided test, so > when alpha = 0.5, there is 0.25 in each tail. > > is wrong. > It looks like JMP is really doing two-sided > test, but if I set Alpha = 0.05, there is 0.05 in > EACH tail, so the real error level for the mean > difference is 0.1 !
Really? How do you know this?
I ask because when I look in JMP Help, in a chapter entitled "Details of Comparison Circles", it sure looks to me like they are claiming that the comparison circles are based upon alpha/2 in each tail.
> This is the second part of my question - is this is a > bug or is it by design?
I created a very small data set, so I can do the calculations by hand. The prob > |t| is very clearly the two-tailed probability.
If you look at the comparison circles in my specific data set, the means are not different at alpha = 0.05. Matches my hand calculations. For this data set, if you set the alpha to exactly equal the prob > |t|, the circles still show no difference. If you increase the alpha to be slightly greater than the prob > |t|, the circles now show a statistically significant difference. This is exactly what you would expect.