F-Test for Linear Relationship

[#1]

Hello

I'm trying to perform an f-test to check if there is a linear relationship between 2 continous variables at the 0.1 level. I've done Fit Y by X -> Fit Line and have my answer of Prob > F = < 0.001. However when I change the alpha level under the linear fit level none of the numbers change. Am I doing something wrong because I thought the ANOVA results should change as alpha changes.

Thanks

[#2 · Accepted solution]

Alpha does not influence the calculation of F and hence not the p-value. It is true that the critical F-value (used in a hypotheses test) changes with alpha but not the estimated F that is compared with the critical F. 

However, if you enable "Confid curve fit" you can see how the confidence bands narrow or widen as alpha is changed.

[#3]

Thanks.

Quick question, how would I be able to find the critical F value in a hypothesis test?

[#4]

The traditional way is to look it up in a table. In JMP there is a function F Quantile( ) that return F for a given p and the two-dimensional degrees of freedom.

For linear regression with two variables, 20 observations and alpha = 0.1 you use the formula

Fcrit = F quantile(0.9, 1, 18);

Show(Fcrit)

Fcrit = 3,00697659179545;

If your F exceeds this Fcrit you can reject the null hypotheses at a the chosen sign level. However you already knew that if you got p < 0,001. In these days with software that calculates the actual p-value for you, critical values is not used as much as before. At least not explicitly.

[#5]

Great thanks.

Just to double check I understand everything here, since p<0.001 we reject the null hypothesis that that data is not linear and accept the alternative hypothesis that there is in fact a linear relationship, right?

[#6]

Well, we do actually not test if the relationship is linear. Linearity is our assumption when using this type of test. The relationship may still be nonlinear, even if p < 0.001. A quick way to identify nonlinearity is to look for any patterns in the residuals.

With a  p<0.001 we can with quite good certainty reject a null hypthesis of slope being zero, i.e. we have a strong (and statistically significant) indication of a positive relationship between x and y. But remember that linear regression by itself cannot "prove" anything in terms of causality. For that we use our scientific understanding of the process (x effects y?  y effects x? Or are there other factors that can influence both).