Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Discussions
- :
- Re: Weighted Kappas

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Weighted Kappas

Jan 5, 2019 1:48 PM
(212 views)

Hi, we have a study where two raters are rating a 50 items for quality on a scale of 1-4. We wanted to do a weighted kappa in JMP; I entered the data, set the data type as ordinal, but JMP seems to only produce a simple Kappa when I Fit Y by X and look at the Agreement Statistic.

Is JMP capable of calculating a weighted kappa?

Thanks in advance!

1 REPLY 1

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Weighted Kappas

I did not find a buit-in solution to your question. Of course JMP is capable, but it needs a script (unless one of the JMP developers has an alternate solution).

You did not report which version of JMP and which operating system you have, so I am not sure this will work for you. Attached is a script that I have tested on JMP 12,13 and 14. It uses methods to be backward compatible with JMP12. It :

- Runs on the current data table and calls up a Contingency analysis. In other words, it uses the JMP UI.
- The script is written for one contingency report, that is, one Y and one X ordinal variable.
- Specifies
**Expected**values to be included in the contingency table to reduce the complexity (and risk) of scripting errors. - Makes a table from the Contingency table.
- From that, it creates the observed, and expected probabilities.
- It calculates the standard kappa where the weight matrix is the Identity matrix; a Linear weight 1- abs(i - j)/(k-1); and a quadratic weight which is 1 - square( (i-j)/(k-1) ). See the references reported in the script.
- The value of k, kL and kQ are appended to the JMP report (see below).
- The calculated kappa is reported as a check since it should match the kappa reported by JMP
- Weighted errors and Confidence Intervals are not calculated in this script.

This might be a wish list item for future JMP versions. Since some references for weighted kappa include more than Linear and Quadratic weights for ordinal categories, you should specify if this wish list item should allow for custom weights.

The script is attached and a sample data set where column **n** should be used as **Freq. **slide58.jmp is a summary table from the third reported reference (page 58). It is one of 10 tables used to test the script. I hope this helps.