Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Discussions
- :
- Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var =...

Topic Options

- Start Article
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Nov 2, 2017 1:01 PM
(2214 views)

I have the following mock data set that I created specifically to test association between two categorical variables (X = BIOMARKER, Y = RESPONSE, Freq = FREQ), grouped by a third categorical variable = ARM

ARM | BIOMARKER | RESPONSE | FREQ |

PBO | BM- | NR | 25 |

PBO | BM- | R | 25 |

PBO | BM+ | NR | 21 |

PBO | BM+ | R | 29 |

ACTIVE | BM- | NR | 38 |

ACTIVE | BM- | R | 12 |

ACTIVE | BM+ | NR | 20 |

ACTIVE | BM+ | R | 30 |

When I perform the X by Y contigency analysis followed by CMH test, I get the following result:

Cochran-Mantel-Haenszel Tests | |||

Stratified by | |||

ARM | |||

CMH Test | ChiSquare | DF | Prob>Chisq |

Correlation of Scores | 9.739 | 1 | 0.0018 |

Row Score by Col Categories | 9.739 | 1 | 0.0018 |

Col Score by Row Categories | 9.739 | 1 | 0.0018 |

General Assoc. of Categories | 9.739 | 1 | 0.0018 |

But when I try the reciprocal analysis where X = ARM, Y = RESPONSE, and Freq = FREQ, grouped by BIOMARKER, I get a different result:

Cochran-Mantel-Haenszel Tests | |||

Stratified by | |||

BIOMARKER | |||

CMH Test | ChiSquare | DF | Prob>Chisq |

Correlation of Scores | 3.0013 | 1 | 0.0832 |

Row Score by Col Categories | 3.0013 | 1 | 0.0832 |

Col Score by Row Categories | 3.0013 | 1 | 0.0832 |

General Assoc. of Categories | 3.0013 | 1 | 0.0832 |

Clearly, I'm not understanding the mechanic of the CMH analysis because I would have expected to get the same results either way.

Also, in this example, there is only contrast in BIOMARKER in the ACTIVE group (by design) but I unsure how to calculate the p value for each ARM (beside using the "By" option).

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Why would you expect the same results from two different analyses?

Pay attention to the *analysis roles*. The **Y** role is for the response or outcome. The **X** role is for the explanatory or predictive variable. Use these roles to define the *association *of interest. The CMH analysis uses a third role: **stratum**. Use this role when you want to see if the *significance* of the association between Y and X is greater or lesser across the strata (levels in the third variable).

Your first analysis explored/tested the association between Response and Biomarker stratified by ARM. Your second analysis explored/tested the association between Response and ARM stratified by Biomarker. These two analyses are not the same.

See: Agresti, Alan (1996) *An Introduction to Categorical Data Analysis*, John Wiley & Sons, New York, page 61.

Learn it once, use it forever!

4 REPLIES 4

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Why would you expect the same results from two different analyses?

Pay attention to the *analysis roles*. The **Y** role is for the response or outcome. The **X** role is for the explanatory or predictive variable. Use these roles to define the *association *of interest. The CMH analysis uses a third role: **stratum**. Use this role when you want to see if the *significance* of the association between Y and X is greater or lesser across the strata (levels in the third variable).

Your first analysis explored/tested the association between Response and Biomarker stratified by ARM. Your second analysis explored/tested the association between Response and ARM stratified by Biomarker. These two analyses are not the same.

See: Agresti, Alan (1996) *An Introduction to Categorical Data Analysis*, John Wiley & Sons, New York, page 61.

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Dear Mark,

Thank you for your clarification. Sorry for my limited understanding of the CMH test but, as a follow up, I would like to know if there are any circumstances where the association between reponse and predictive variable would be non-significant but the CMH test would be significant across the strata?

So far, I have experimented with some mock data and I can't find a combination of data producing this type of output.

Thank you for your help.

Sincerely,

Thierry

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

I was unable to create an example of such a case and I don't have access to Agresti's textbook to check the computation and determine under what circumstances, if any, it might be possible. So you will have to wait for a while for an answer from me or perhaps another expert might answer in the meantime.

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Your hunch is correct. A colleague of mine, Bob Lucas, replied with the following answer:

"I went to this link.

From looking at the test statistic, I do not think it is possible.

The numerator of the CMH test is just the sum over all the strata of the observed minus predicted squared. The denominator is just the scaling factor so that the distribution is asymptotically Chi Square.

The numerator of the CMH test is similar to the numerator of the F in ANOVA. Consequently, I think it is a similar situation to overall F test in ANOVA vs. comparing two means in an ANOVA.

I think it is possible that in one stratum, there may be a significant association but none in the others so by averaging over all the strata, the one association is hidden. Similar to how two means in an ANOVA can be different but the overall F is not significant.

In ANOVA, if none of the pairwise mean comparison are significant, the overall F cannot be."

Learn it once, use it forever!