Share your ideas for the JMP Scripting Unsession at Discovery Summit by September 17th. We hope to see you there!
Choose Language Hide Translation Bar
Highlighted
mgerusdurand
Level I

Kappa Score

Bonjour,

 

Malgrès mes recherches je n'ai pas trouver réponse à mon problème donc le voici:

Il me faut calculer un kappa score sur une table de contingence. J'ai bien les mêmes catégories pour les 2 valeurs mais JMP 15 ne veut pas me calculer le Kappa. Pourtant la formule de Kappa n'a pas de restriction étant donné qui s'agit du ratio de la différence de probabilité entre réelle et attendue sur l'inverse de la probabilité attendue.

 

Pourriez-vous m'aider svp?

 

Merci d'avance 

Annotation 2020-08-27 101120.jpg

Annotation 2020-08-27 101058.jpg

MGD
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Kappa Score

I would not really interpret kappa in this case. The issue is that value 1 is ALWAYS 3. There is no variability with that value 1. There are several reasons that could happen, but that is the issue. Statistics work with variability and there is no variability there.

Dan Obermiller

View solution in original post

5 REPLIES 5
Highlighted

Re: Kappa Score

Actually, JMP did calculate Kappa for you: it is 0.

 

To see this, let's walk through the calculation.

 

Pr(a) = Probability of agreement = Agreeing on value 1 + Agreeing on value 2 + Agreeing on value 3 = 0/30 + 0/30 + 29/30 = 0.967.

 

Pr(e) = Probability expected (you use the marginal totals here) = Expected value 1 + Expected value 2 + Expected value 3 = 0/30 * 1/30 + 0/30 * 0/30 + 29/30 * 30/30 = 0.967.

 

kappa = 0 / (1-.967) = 0.

Dan Obermiller
Highlighted
mgerusdurand
Level I

Re: Kappa Score

Bonjour,

 

Merci pour cette réponse. Oui en effet en regardant mieux le Kappa est bien de 0.

Mais alors je trouve ca étrange. Cela veut dire que mes 97% de concordance sont juste dû au hasard?

De premier abord je dirai que d'avoir 29 fois la même réponse sur 30 est plutôt supérieur au hasard non? 

Y'a t-il alors un meilleur test de concordance à faire pour ces données?

 

Merci encore pour votre aide.

 

Marie 

MGD
Highlighted

Re: Kappa Score

I would not really interpret kappa in this case. The issue is that value 1 is ALWAYS 3. There is no variability with that value 1. There are several reasons that could happen, but that is the issue. Statistics work with variability and there is no variability there.

Dan Obermiller

View solution in original post

Highlighted

Re: Kappa Score

One additional thought: You could say that Value 2 will give a result of 3 29/30 times. Just ignore Value 1 since it is always a 3.

Dan Obermiller
Highlighted
mgerusdurand
Level I

Re: Kappa Score

Thanks a lot Dan,

Usually scores are dispatched from 0 to 3 it is why I decided to applied Kappa, but in this case I guess I will do as you said and just talk about overall concordance as Kappa is not applicable here.

Best
MGD
Article Labels