Author
Melvin Alexander, Operations Research Analyst, Social Security Administration MelA@ssa
Fleiss' kappa (in the JMP Attribute Gauge platform) using ordinal rating scales helped assess inter-rater agreement between independent radiologists who diagnosed patients with penetrating abdominal injuries. One drawback of Fleiss' kappa is that it does not estimate inter-rater reliability well enough since it is limited to disagreement distributions between raters being treated equally. Typically, disagreements are not all alike and have different magnitudes with multiple raters. When compared to Fleiss' kappa, Krippendorff's alpha better differentiates between raters' disagreements for various sample sizes; and estimates judgments, with or without missing data, across multiple measurement scales (binary, nominal, ordinal, interval and ratio) for multiple raters. Currently, Krippendorff's alpha is not available in JMP, but is computed in the R open-source statistical programming language. JMP connects to R via JSL to execute R commands and exchange data. This presentation will demonstrate JMP and R integration that allows users to take advantage of the powerful capabilities in both tools. Combining JMP and R helps users gain more insight and get better analytic results. Results helped radiologists discern which imaging signs detected injuries from signs that needed improved detection training.
