Hi all
I'm working with data from some pass-fail testing, where a difference in performance has been observed for some samples grouped into left and right. I'm trying to analyse whether this is a real difference or rather variation due to statistical variance. I'm doing this by using the Contingency analysis (Fit X by Y) framework in JMP.
The issue is that I'm a bit confused by the results, which are shown below. Assuming a confidence of 95%, the Chi-Square test shows that the ratio of passed and failed devices does not appear to be dependent on if they are left or right samples.
Now if i also perform a Two Sample Test for Proportions to test if the probability off passes for right side samples will be lower than or equal to the number of left side passes, i find that i can reject this hypothesis, implying that passes for right side samples are higher.
I know that these two tests test different type of hypothesis, but it seems to me that these to conclusions contradict each other.
Finally, looking at the two sample test I find that the confidence interval goes from -0.7% to 9.4% for the difference P(pass|right)-P(pass|left), which surprising to me appear to contradict the conclusion drawn for the two sample test just above. Shouldn't all values in this interval be above or equal to zero?
All comments and suggestions are welcome to help me understand all these great tools.