cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
JMP is taking Discovery online, April 16 and 18. Register today and join us for interactive sessions featuring popular presentation topics, networking, and discussions with the experts.
Choose Language Hide Translation Bar
bns
bns
Level II

Variability in Responses

Hi All,

 

I am trying to analyze a data set in which 15 doctors all ranked images of a disease on a scale of 0-9. There were 110 images. I want to look at the variability of the doctors/determine how good they are at coming to a similar ranking. Any idea on how to go about this?

2 ACCEPTED SOLUTIONS

Accepted Solutions
P_Bartell
Level VIII

Re: Variability in Responses

The Distribution platform is the 'go to' platform for looking at each response one at a time. Fit Y by X could be very useful for looking across the evaluators. Much can also be done in Graph Builder with things like data filters. All good analysis starts with a clear articulation  of the practical problems you would like to address...then find the appropriate analysis methods to answer those questions.

View solution in original post

Re: Variability in Responses

Adding to Peter's advice, you can analyze your data using the Contingency platform. First be sure that the Rating response is set to use the Ordinal modeling type. The Rater data column that identifies the doctor should use the Nominal modeling type. Select Analyze > Fit Y by X. Assign Rating to the Y role and Rater to the X role and click OK. Click the red triangle next to Contingency and select Agreement Statistic. The kappa statistic indicates the strength of the agreement. Kappa close to zero indicates poor agreement and kappa close to one indicates good agreement.

View solution in original post

4 REPLIES 4
P_Bartell
Level VIII

Re: Variability in Responses

The Distribution platform is the 'go to' platform for looking at each response one at a time. Fit Y by X could be very useful for looking across the evaluators. Much can also be done in Graph Builder with things like data filters. All good analysis starts with a clear articulation  of the practical problems you would like to address...then find the appropriate analysis methods to answer those questions.

Re: Variability in Responses

Adding to Peter's advice, you can analyze your data using the Contingency platform. First be sure that the Rating response is set to use the Ordinal modeling type. The Rater data column that identifies the doctor should use the Nominal modeling type. Select Analyze > Fit Y by X. Assign Rating to the Y role and Rater to the X role and click OK. Click the red triangle next to Contingency and select Agreement Statistic. The kappa statistic indicates the strength of the agreement. Kappa close to zero indicates poor agreement and kappa close to one indicates good agreement.

bns
bns
Level II

Re: Variability in Responses

Thanks for the advice! So I was able to get contingency tables, but do not see agreement statistics. I posted a file of the two options that come up when clicking the red triangle. One is for the red triangle for contingency analysis, and one is for contigency table. 

Screen Shot 2019-07-27 at 2.21.26 PM.png

 

Screen Shot 2019-07-27 at 2.21.17 PM.png

Hope i'm not being confusing! I appreciate the help i've received. 

bns
bns
Level II

Re: Variability in Responses

Nevermind. For some reason, when I ran it again, it worked as you said. Thanks!