BookmarkSubscribe
Choose Language Hide Translation Bar
Community Trekker

## Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different results

I have the following mock data set that I created specifically to test association between two categorical variables (X = BIOMARKER, Y = RESPONSE, Freq = FREQ), grouped by a third categorical variable = ARM

 ARM BIOMARKER RESPONSE FREQ PBO BM- NR 25 PBO BM- R 25 PBO BM+ NR 21 PBO BM+ R 29 ACTIVE BM- NR 38 ACTIVE BM- R 12 ACTIVE BM+ NR 20 ACTIVE BM+ R 30

When I perform the X by Y contigency analysis followed by CMH test, I get the following result:

 Cochran-Mantel-Haenszel Tests Stratified by ARM CMH Test ChiSquare DF Prob>Chisq Correlation of Scores 9.739 1 0.0018 Row Score by Col Categories 9.739 1 0.0018 Col Score by Row Categories 9.739 1 0.0018 General Assoc. of Categories 9.739 1 0.0018

But when I try the reciprocal analysis where X = ARM, Y = RESPONSE, and Freq = FREQ, grouped by BIOMARKER, I get a different result:

 Cochran-Mantel-Haenszel Tests Stratified by BIOMARKER CMH Test ChiSquare DF Prob>Chisq Correlation of Scores 3.0013 1 0.0832 Row Score by Col Categories 3.0013 1 0.0832 Col Score by Row Categories 3.0013 1 0.0832 General Assoc. of Categories 3.0013 1 0.0832

Clearly, I'm not understanding the mechanic of the CMH analysis because I would have expected to get the same results either way.

Also, in this example, there is only contrast in BIOMARKER in the ACTIVE group (by design) but I unsure how to calculate the p value for each ARM (beside using the "By" option).

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Staff

## Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Why would you expect the same results from two different analyses?

Pay attention to the analysis roles. The Y role is for the response or outcome. The X role is for the explanatory or predictive variable. Use these roles to define the association of interest. The CMH analysis uses a third role: stratum. Use this role when you want to see if the significance of the association between Y and X is greater or lesser across the strata (levels in the third variable).

Your first analysis explored/tested the association between Response and Biomarker stratified by ARM. Your second analysis explored/tested the association between Response and ARM stratified by Biomarker. These two analyses are not the same.

See: Agresti, Alan (1996) An Introduction to Categorical Data Analysis, John Wiley & Sons, New York, page 61.

Learn it once, use it forever!
4 REPLIES 4
Staff

## Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Why would you expect the same results from two different analyses?

Pay attention to the analysis roles. The Y role is for the response or outcome. The X role is for the explanatory or predictive variable. Use these roles to define the association of interest. The CMH analysis uses a third role: stratum. Use this role when you want to see if the significance of the association between Y and X is greater or lesser across the strata (levels in the third variable).

Your first analysis explored/tested the association between Response and Biomarker stratified by ARM. Your second analysis explored/tested the association between Response and ARM stratified by Biomarker. These two analyses are not the same.

See: Agresti, Alan (1996) An Introduction to Categorical Data Analysis, John Wiley & Sons, New York, page 61.

Learn it once, use it forever!
Community Trekker

## Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Dear Mark,

Thank you for your clarification. Sorry for my limited understanding of the CMH test but, as a follow up, I would like to know if there are any circumstances where the association between reponse and predictive variable would be non-significant but the CMH test would be significant across the strata?

So far, I have experimented with some mock data and I can't find a combination of data producing this type of output.

Sincerely,

Thierry

Staff

## Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

I was unable to create an example of such a case and I don't have access to Agresti's textbook to check the computation and determine under what circumstances, if any, it might be possible. So you will have to wait for a while for an answer from me or perhaps another expert might answer in the meantime.

Learn it once, use it forever!
Staff

## Re: Cochran-Mantel-Haenszel Tests Interpretation: permuting X var and Blocking Var = different resul

Your hunch is correct. A colleague of mine, Bob Lucas, replied with the following answer: