cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
mgerusdurand
Level IV

Kappa Score

Bonjour,

 

Malgrès mes recherches je n'ai pas trouver réponse à mon problème donc le voici:

Il me faut calculer un kappa score sur une table de contingence. J'ai bien les mêmes catégories pour les 2 valeurs mais JMP 15 ne veut pas me calculer le Kappa. Pourtant la formule de Kappa n'a pas de restriction étant donné qui s'agit du ratio de la différence de probabilité entre réelle et attendue sur l'inverse de la probabilité attendue.

 

Pourriez-vous m'aider svp?

 

Merci d'avance 

Annotation 2020-08-27 101120.jpg

Annotation 2020-08-27 101058.jpg

MGD
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Kappa Score

I would not really interpret kappa in this case. The issue is that value 1 is ALWAYS 3. There is no variability with that value 1. There are several reasons that could happen, but that is the issue. Statistics work with variability and there is no variability there.

Dan Obermiller

View solution in original post

5 REPLIES 5

Re: Kappa Score

Actually, JMP did calculate Kappa for you: it is 0.

 

To see this, let's walk through the calculation.

 

Pr(a) = Probability of agreement = Agreeing on value 1 + Agreeing on value 2 + Agreeing on value 3 = 0/30 + 0/30 + 29/30 = 0.967.

 

Pr(e) = Probability expected (you use the marginal totals here) = Expected value 1 + Expected value 2 + Expected value 3 = 0/30 * 1/30 + 0/30 * 0/30 + 29/30 * 30/30 = 0.967.

 

kappa = 0 / (1-.967) = 0.

Dan Obermiller
mgerusdurand
Level IV

Re: Kappa Score

Bonjour,

 

Merci pour cette réponse. Oui en effet en regardant mieux le Kappa est bien de 0.

Mais alors je trouve ca étrange. Cela veut dire que mes 97% de concordance sont juste dû au hasard?

De premier abord je dirai que d'avoir 29 fois la même réponse sur 30 est plutôt supérieur au hasard non? 

Y'a t-il alors un meilleur test de concordance à faire pour ces données?

 

Merci encore pour votre aide.

 

Marie 

MGD

Re: Kappa Score

I would not really interpret kappa in this case. The issue is that value 1 is ALWAYS 3. There is no variability with that value 1. There are several reasons that could happen, but that is the issue. Statistics work with variability and there is no variability there.

Dan Obermiller

Re: Kappa Score

One additional thought: You could say that Value 2 will give a result of 3 29/30 times. Just ignore Value 1 since it is always a 3.

Dan Obermiller
mgerusdurand
Level IV

Re: Kappa Score

Thanks a lot Dan,

Usually scores are dispatched from 0 to 3 it is why I decided to applied Kappa, but in this case I guess I will do as you said and just talk about overall concordance as Kappa is not applicable here.

Best
MGD