topic Re: Test of agreement for presence/absence by month in Discussions

Test of agreement for presence/absence by month

fishguy — Mon, 02 Apr 2018 16:57:34 GMT

Hi,

I collected data on the monthly occurrence of a parasite and the monthly occurrence of a histological lesion in the tissues where the parasite infects. The parasite is strongly seasonal - peak prevalence and intensities occur in March and April. The lesions are more spread out throughout the year but appear to be more common in those months. So for each (lesion or parasite), I can construct a contingency table (2x12) to demonstrate the months where the prevalence of lesions and parasites are highest. What I would like to do is test how closely the parasites and lesions agree. Is there a test of agreement to compare monthly lesion and parasite prevalence?

I can disregard month and construct a contingency table of parasite and lesion. The results show a Kappa of 0.0248 with a P = 0.5804. Also, Bowker's test for symmetry (greater than 2x2 contingency table) indicates symmetry of disagreement chisq =22.6, P <0.001. I would take these results that there is no strong measure of agreement between the lesion and parasite overall. However, this doesn't account for month.

Re: Test of agreement for presence/absence by month

Mark_Bailey — Mon, 02 Apr 2018 17:11:06 GMT

Agreement is for Y and X with the same scale. Your description mentions counts (lesions, parasites) but not the scales used for either one.

Association is more general. You could stratefy your contingency table by Month. One of the CMH tests should cover your hypothesis.

Re: Test of agreement for presence/absence by month

fishguy — Mon, 02 Apr 2018 17:31:07 GMT

Hi,

I recorded parasite and lesion presence or absence for each fish. The
monthly proportion of fish with each ranges from low teens to 100% so If I
understand correctly, they are both on the same scale

What are CMH tests and how do I stratify by month.

Thanks!
Mark

Re: Test of agreement for presence/absence by month

fishguy — Mon, 02 Apr 2018 18:19:23 GMT

Ok, I think it is the Cochran Mantel Haenzsel test you're referring too. I found an example and understand how to run it now. So I chose the presence/absence of lesion and parasite as the X and Y and then in the pop-up window chose month as the grouping variable. Is the default that the null hypothesis is no correlation between parasite and lesion?

Thanks!

Re: Test of agreement for presence/absence by month

Mark_Bailey — Mon, 02 Apr 2018 19:25:06 GMT

You are way ahead of me! Yes, the null is no association. You test the overall association (unstratified) first with the Contingency platform, then select CMH from the red triangle for stratification by Month.

Re: Test of agreement for presence/absence by month

fishguy — Wed, 04 Apr 2018 13:32:32 GMT

Mark,

I tried running the CMH test with parasite and lesion as X and Y stratified by month. Correlation scores are blank in the results box followed by a message that the statistic was not computed as the covariance matrix was singular. What does that mean?

Thanks!

Re: Test of agreement for presence/absence by month

fishguy — Wed, 04 Apr 2018 13:46:58 GMT

Mark,

I have another question or 2. I also ran general agreement statistics between the parasite and lesion, ignoring month. The Kappa score was high (0.66) and was significant P<0.001. Bowker's test was also significant chisq 5.4, P=0.196. Does Bowker's test confirm the Kappa? I'm not sure how to take Bowker's result as it seems like it is testing disagreement rather than agreement. Can you explain?

Also, after thinking about this a bit, I don't know that it matters that I include month in the analysis. Perhaps Kappa and Bowker's would be sufficient. However, I would like to understand if the CMH test provides any additional useful information beyond Kappa and Bowkers.

Thanks again!!

Re: Test of agreement for presence/absence by month

Mark_Bailey — Fri, 06 Apr 2018 11:12:47 GMT

Singularity usually means that the model behind this test is over-specified. It would be like including a quadratic term in the linear predictor when the X variable has only two levels. Such data does not support estimating the model. Can you post the Contingency platform with your data before and after applying the CMH tests?

Re: Test of agreement for presence/absence by month

Mark_Bailey — Fri, 06 Apr 2018 11:25:27 GMT

Kappa measures agreement between raters or ratings. For McNemar's / Bowker's test of symmetry, the null hypothesis is that the probabilities in the square table satisfy symmetry or that p_ij = p_ji for all pairs of table cells.

The CMH tests are about association and the choice among the four alternatives depends on the modeling type of the X and Y variables. From JMP Help:

The Cochran-Mantel-Haenszel test discovers if there is a relationship between two categorical variables after blocking across a third classification.
Correlation of Scores: Applicable when both Y or X are ordinal or interval. The alternative hypothesis is that there is a linear association between Y and X in at least one level of the blocking variable.
Row Score by Col Categories: Applicable when Y is ordinal or interval. The alternative hypothesis is that, for at least one level of the blocking variable, the mean scores of the r rows are unequal.
Col Score by Row Categories: Applicable when X is ordinal or interval. The alternative hypothesis is that, for at least one level of the blocking variable, the mean scores of the c columns are unequal.
General Assoc. of Categories: Tests that for at least one level of the blocking variable, there is some type of association between X and Y.

The test statistic is essentially computed for each stratum and combined into the overall test statistic.

Re: Test of agreement for presence/absence by month

fishguy — Fri, 06 Apr 2018 12:48:46 GMT

Here is the contingency table and CMH results. I set lesion and parasite presence/absence as ordinal and it worked. I stratified by month. I think I had my columns set as nominal previously.

I believe the null hypothesis is that there is no association between lesions and parasites. Therefore it looks like I have a highly significant association between them. So what I'm wondering is what added information does the CMH and stratification provide above that of the standard ChiSquare?

Thanks for your help!

Re: Test of agreement for presence/absence by month

Mark_Bailey — Fri, 06 Apr 2018 14:19:18 GMT

I don't believe that using the nominal modeling type would cause a singularity in the CMH tests, especially since you have only two levels for X and for Y.

Yes, the null hypothesis is no association between X and Y.

Yes, the unstratified contingency analysis is highly significant (LRT chi square > 65, p-value < 0.0001). (Note that chi square cannot be used to assess the strength of the association, only its significance.)

Yes, the relevant CMH test (General Association of Categories) is highly significant (chi square > 40, p-value < 0.0001). The stratification helps assure that the association is consistently present across the strata, if that question must be answered. The CMH test also somewhat relaxes the sample size requirement for the chi square statistic, which is not an issue in this case. (No more than 20% of cells with expected count less than 5.)

Re: Test of agreement for presence/absence by month

fishguy — Fri, 06 Apr 2018 15:35:02 GMT

Thanks for your help!