topic Re: Agreement statistics with continuous variable in Discussions

Agreement statistics with continuous variable

jmeredith09 — Fri, 09 Jun 2023 00:53:23 GMT

Hello,

I have a observation scale that calculate amount of times a behavior occurs which could be any where from 0 to 40+ so I assume it could only be a continuous variable. I am wanting to run agreement statistic but because both are continuous it won't let me do that in JMP. Any tips or tricks on how to get an agreement stats with continuous variables?

Re: Agreement statistics with continuous variable

Mark_Bailey — Thu, 04 Aug 2022 13:48:26 GMT

You're correct. Agreement, a stricter form of association, is only meaningful with categorical data (nominal or ordinal modeling type in JMP). Correlation is the meaningful way to assess the linear relationship between continuous variables.

On the other hand, your data is counting the times a behavior occurs. This response is usually modelled with a Poisson distribution in a generalized linear model. The linear predictor includes factors that might account for changes in the counts.

Let's step back and think about what you are really trying to do. What information are you hoping to gain? What kind of comparison are you making? What are your hypotheses?

Re: Agreement statistics with continuous variable

jmeredith09 — Thu, 04 Aug 2022 15:44:05 GMT

This is helpful thank you, so what I am wanting is interrater agreement? It is the same coding tool but used between two different coders and so I am looking to see if the tool is reliable between coders? Is that helpful?

Re: Agreement statistics with continuous variable

P_Bartell — Thu, 04 Aug 2022 17:29:05 GMT

I think what you may be after then is the coefficient of concordance. I don't think JMP supports that out of the box...but maybe somebody built an add in or script? There are some non parametric measures of concordance in JMP but I don't think they are optimal for what you are trying to do. At the very least you can start with a Fit Y by X visual assessment to give you some level of information rather relying on a mononumerotic test statistic of hypothesis test result.

Re: Agreement statistics with continuous variable

Mark_Bailey — Thu, 04 Aug 2022 17:49:47 GMT

In case you missed it, please review this chapter in JMP Help about Measurement System Analysis. It provides an overview of the topic and the capabilities of JMP.

When there is a measure for something with continuous data (e.g., correlation), there is generally an analog for it in the other type of data (e.g., association, agreement). So I hope we can identify something that already exists and avoid scripting a solution.

Re: Agreement statistics with continuous variable

statman — Fri, 05 Aug 2022 20:42:39 GMT

Kappa statistic? I don't know what a coding tool is? Can the differences be quantified?