Choose Language Hide Translation Bar
Highlighted
bns
bns
Level I

Variability in Responses

Hi All,

 

I am trying to analyze a data set in which 15 doctors all ranked images of a disease on a scale of 0-9. There were 110 images. I want to look at the variability of the doctors/determine how good they are at coming to a similar ranking. Any idea on how to go about this?

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted
P_Bartell
Level VI

Re: Variability in Responses

The Distribution platform is the 'go to' platform for looking at each response one at a time. Fit Y by X could be very useful for looking across the evaluators. Much can also be done in Graph Builder with things like data filters. All good analysis starts with a clear articulation  of the practical problems you would like to address...then find the appropriate analysis methods to answer those questions.

View solution in original post

Highlighted

Re: Variability in Responses

Adding to Peter's advice, you can analyze your data using the Contingency platform. First be sure that the Rating response is set to use the Ordinal modeling type. The Rater data column that identifies the doctor should use the Nominal modeling type. Select Analyze > Fit Y by X. Assign Rating to the Y role and Rater to the X role and click OK. Click the red triangle next to Contingency and select Agreement Statistic. The kappa statistic indicates the strength of the agreement. Kappa close to zero indicates poor agreement and kappa close to one indicates good agreement.

Learn it once, use it forever!

View solution in original post

4 REPLIES 4
Highlighted
P_Bartell
Level VI

Re: Variability in Responses

The Distribution platform is the 'go to' platform for looking at each response one at a time. Fit Y by X could be very useful for looking across the evaluators. Much can also be done in Graph Builder with things like data filters. All good analysis starts with a clear articulation  of the practical problems you would like to address...then find the appropriate analysis methods to answer those questions.

View solution in original post

Highlighted

Re: Variability in Responses

Adding to Peter's advice, you can analyze your data using the Contingency platform. First be sure that the Rating response is set to use the Ordinal modeling type. The Rater data column that identifies the doctor should use the Nominal modeling type. Select Analyze > Fit Y by X. Assign Rating to the Y role and Rater to the X role and click OK. Click the red triangle next to Contingency and select Agreement Statistic. The kappa statistic indicates the strength of the agreement. Kappa close to zero indicates poor agreement and kappa close to one indicates good agreement.

Learn it once, use it forever!

View solution in original post

Highlighted
bns
bns
Level I

Re: Variability in Responses

Thanks for the advice! So I was able to get contingency tables, but do not see agreement statistics. I posted a file of the two options that come up when clicking the red triangle. One is for the red triangle for contingency analysis, and one is for contigency table. 

Screen Shot 2019-07-27 at 2.21.26 PM.png

 

Screen Shot 2019-07-27 at 2.21.17 PM.png

Hope i'm not being confusing! I appreciate the help i've received. 

Highlighted
bns
bns
Level I

Re: Variability in Responses

Nevermind. For some reason, when I ran it again, it worked as you said. Thanks!

Article Labels

    There are no labels assigned to this post.