Level: Beginner
Karen Copeland, Owner, Boulder Statistics
Tarek Zikry, Technical Intern, SAS
Mia Stephens, JMP Academic Ambassador, SAS
For dichotomous classification models (good versus bad, negative versus positive, yes versus no, etc.), an ROC curve and the associated area under the curve (AUC) are often used to describe the ability of the model to differentiate the two groups. While this is a start, the ROC curve can be hard to understand, and the AUC can be misleading outside the context of the modeling problem. For some models, the overall misclassification rate is key; for other models, there is a cost associated with misclassifications. This cost can be very different for a false negative versus a false positive. We will discuss dichotomous models for prediction and their associated ROC curves, lift curves and alternative displays of model performance. We will also discuss methods for selecting optimal cutoffs for classification and measures of performance based on the specified cutoff. To facilitate the selection of the optimal cutoff for classification models and the calculation of performance metrics, we have developed an interactive and graphical JMP application. We will demonstrate this application with a compelling case study.
![2018 Discovery Beyond ROC Curves_Page_01.jpg 2018 Discovery Beyond ROC Curves_Page_01.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13521i016C6498C37407D0/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_02.jpg 2018 Discovery Beyond ROC Curves_Page_02.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13519iD663398AB85ACA88/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_03.jpg 2018 Discovery Beyond ROC Curves_Page_03.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13522iBA2F56E1C6D9BEFC/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_04.jpg 2018 Discovery Beyond ROC Curves_Page_04.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13520i091F3E7133235F75/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_05.jpg 2018 Discovery Beyond ROC Curves_Page_05.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13523i52C8B17A30F913C3/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_06.jpg 2018 Discovery Beyond ROC Curves_Page_06.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13524i81B25BB88830FC75/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_07.jpg 2018 Discovery Beyond ROC Curves_Page_07.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13525iE25C70C9D671E1A9/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_08.jpg 2018 Discovery Beyond ROC Curves_Page_08.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13526i593B711D7BBC1CE0/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_09.jpg 2018 Discovery Beyond ROC Curves_Page_09.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13527i6917763A1B256C57/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_10.jpg 2018 Discovery Beyond ROC Curves_Page_10.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13528i344C02E16535610F/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_11.jpg 2018 Discovery Beyond ROC Curves_Page_11.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13529i7FE44FEA6E9B1F90/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_12.jpg 2018 Discovery Beyond ROC Curves_Page_12.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13530i672FA1225B616BFD/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_13.jpg 2018 Discovery Beyond ROC Curves_Page_13.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13531i6559A61895568952/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_14.jpg 2018 Discovery Beyond ROC Curves_Page_14.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13532iCD84B17BA8426E96/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_15.jpg 2018 Discovery Beyond ROC Curves_Page_15.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13533i6F911E08BC282AAC/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_16.jpg 2018 Discovery Beyond ROC Curves_Page_16.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13534i7E268DB9A79B1FD5/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_17.jpg 2018 Discovery Beyond ROC Curves_Page_17.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13535i10F6ECACB3F0116A/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_18.jpg 2018 Discovery Beyond ROC Curves_Page_18.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13536iD575E70CC9D1DE83/image-size/medium?v=v2&px=400)
![2018 Discovery Beyond ROC Curves_Page_19.jpg 2018 Discovery Beyond ROC Curves_Page_19.jpg](https://community.jmp.com/t5/image/serverpage/image-id/13537i95E1C233CF317D21/image-size/medium?v=v2&px=400)