Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Search instead for
Did you mean:
World Statistics Day was yesterday, but we’re celebrating all week long! This celebration means acknowledging the impact statistics has on our world. Who is your favorite statistician? Share with us who they are and why they top your favorites list.
Help with Nominal Logistic Fit: Confusion Matrix vs ROC Table Output?
Sep 21, 2018 9:09 AM(5266 views)
Hi JMP Community,
I have been studying the Nominal Logistic Fit to determine the value of a Baseline Biomarker to predict the outcome of a Clinical Treatment.
I thought that I understood the concept of the Confusion Matrix: it returns the numbers of True Positive, True Negative, False Positive, and False Negative for a given model for the Training data set and, if defined, the Validation set. However, when I compare the Confusion Matrix to the best outcome from the ROC Table (Maximum SENSITIVITY - (1 - SPECIFICITY) value), I struggle to reconcile the two.
For example I have a a model with a ROC AUC = 0.654 (rather weak association) where the Confusion Matrix returns:
--> which is really bad (actually worst than expected for the ROC AUC value).
For the same model, the ROC Table best combination of SENSITIVITY and SPECIFICITY is:
SENS - (1-SPEC)
--> which is quite bad but more in line with expected outcome of a model with a ROC AUC = 0.654
So, my questions are:
What is the main difference between Confusion Matrix and the "best" row of the ROC Table?
Is it because the former use the highest Probability and the latter uses the best SENSITIVITY and SPECIFICITY combination?
If I were to present these results, what would be the best option to present the Positive Predictive Value and the Negative Predictive Value?