topic Re: Fischer's exact test odds ratio and Confidence intervals in Discussions

Fischer's exact test odds ratio and Confidence intervals

Srishti — Fri, 09 Jun 2023 00:16:29 GMT

Hi,

I am a regular user of JMP, but without experience in scripting. I have used Fischer's exact test on my analysis, and am getting the results as posted. I have 2 questions:

1. Why do I get 2 difference exact tests and p-values? Which is the one to use?

2. In the odds ratio, the confidence interval does not match the p-value (which is <0.05). I am guessing it has something to do with the way with the CIs are calculated. Can you advise the best way to calculate the exact CI here? I am not experiencing this issue with other exact analyses used in the same dataset

Thanks

Re: Fischer's exact test odds ratio and Confidence intervals

Mark_Bailey — Thu, 14 May 2020 16:43:36 GMT

You do not understand the exact test for a 2x2 contingency table. It is possible to use it three ways. The Alternative Hypothesis explains each way and allows you to decide which of the three tests is appropriate for your decision.
The odds ratio confidence interval does match the tests. The p-values are greater than alpha = 0.05 and the 95% confidence interval for the odds ratio includes 1.

I have no idea what you might have done wrong or differently with this analysis compared to other analyses with the same data set. What other analysis is there?

I do not get the same result for the analysis of the same data. See my result:

Re: Fischer's exact test odds ratio and Confidence intervals

Thierry_S — Thu, 14 May 2020 17:36:00 GMT

Hi Mark,
It looks like the first analysis used the Weight for the Frequency while you used the Counts; would that be a possible source of different behavior?
Best,
TS

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Thu, 14 May 2020 17:47:39 GMT

Thanks for your replies. I also think the 'weight' (which is a sampling weight for the study design) may have changed things, as @Thierry_S suggests. If I only use the 'counts' from this 2*2 table in the 'frequency' section of the contingency platform, I am getting the same results as you, @Mark_Bailey . As for the alternative hypothesis, I would need to use the one that corresponds with two-tailed values only

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Thu, 14 May 2020 17:50:30 GMT

To add to the reply above, these weights are for survey non-response, and there is no complex study design (such as stratification or clustering) that I need to adjust for.

Re: Fischer's exact test odds ratio and Confidence intervals

Mark_Bailey — Thu, 14 May 2020 17:52:17 GMT

Yeah.

Re: Fischer's exact test odds ratio and Confidence intervals

Mark_Bailey — Thu, 14 May 2020 17:54:19 GMT

Why are weights for non-responders used? How are the weights determined?

Using the weight role will change the result of the analysis.

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Thu, 14 May 2020 18:02:17 GMT

This was a survey with ~50% response rate with differences between responders and non-responders, thus our decision to adjust for non-response. Weights were calculated on the basis of 3 baseline characteristics that affected response. Weight calculation was done as follows: for each combination of the 3 variables, the total number with that combination in the original cohort and was divided by the number of patients with that combination who completed a survey.

Re: Fischer's exact test odds ratio and Confidence intervals

Mark_Bailey — Thu, 14 May 2020 19:11:23 GMT

How do you know "differences between responders and non-responders" if the non-responders did not respond? I better let someone who is more familiar with this kind of analysis take over!

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Thu, 14 May 2020 19:14:40 GMT

Differences were with respect to demographic characteristics that we had for all people that were given he survey, eg, age, sex. Hence, we could use these data for comparison

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Tue, 26 May 2020 19:08:39 GMT

Hello @Mark_Bailey . I was wondering if you had any additional insights to this. I haven't found a solution to this problem yet.

I tried making contingency tables (with the numbers I get after weighting)- this seems to give me OR and 2-tailed p-value for the exact test which are consistent with each other. I ran these for several analyses, which seem to give similar but not exactly the same results compared to when the weighting variable is used.

Re: Fischer's exact test odds ratio and Confidence intervals

Mark_Bailey — Tue, 26 May 2020 19:39:37 GMT

I do not have access to JMP computer codes. I recommend that you send your question about the effect of weighting the contingency table analysis to JMP Technical Support (support@jmp.com). Please report back here what you learn from them so that other Community members may learn from your experience.

Re: Fischer's exact test odds ratio and Confidence intervals

Srishti — Fri, 05 Jun 2020 16:32:57 GMT

I reached out to JMP support. The issue was that the weights used were non-integer weights. On using rounded integer weights, the discrepancy in the Odds ratio CIs and p-values disappeared.

Unfortunately, it seems JMP cannot handle non-integer weights. It would be useful to have this feature !