These are instructions on how to download and run the Model Classification Explorer Add-In I created as a JMP Intern this summer with @KarenC and @mia_stephens. This add-in provides a unified dashboard to visualize model cutoffs and error trade-offs simultaneously. You can interactively change a model threshold and immediately see the results propagated in performance measures, confusion matrices, and ROC curves.
NOTE: This may be slow or freeze up with very large data sets.
Step 1: Download the attached .jmpaddin file and example data table.
Data Selection Dialog
Step 1: Cast the Prob[Donut] to X and the Consumption to Y. These two columns are required.
Step 2: Set your target level to “Donut,” as that’s what the model is predicting.
Step 3: Set your alpha level for the statistical analysis, which is left to 0.05 on default.
Step 4: You have the option in the top right to choose the Performance Terminology based on a given application field, which is a purely visual option for the labels of the different measures. In this case, we’ll leave it on “General.”
Step 5: The visual accessibility check box is a feature to change graphical output to be able to interpret results without needing to distinguish colors on the graphs, which we’ll leave off for now.
Step 6: With all these initial parameters set, click “OK” to launch the platform. Depending on the size of your dataset and whether or not you used a validation column, this may take a few seconds to launch.
Interactively Exploring Cutoffs
Let me know if you have any comments or suggestions!