Confidence curves

[#1]

In the paper titled "Confidence curves: an alternative to null hypothesis significance testing for the comparison of clas...", Daniel Berrar details issues related to the use of null hypothesis significance testing for the comparison of classifiers and proposes the use of confidence curves as an alternative. After installing the add-in, you can generate confidence curves for comparing various classifiers with a reference classifier in the following manner:

1. In JMP Pro's menu bar, select "Analyze => Predictive Modeling => Modeling Screening".
2. Select appropriate parameters for the "Y, Response" and "X, Factor" fields.
3. In the "Folded Crossvalidation" section of the dialog, enable the "K Fold Crossvalidation" option and set the number of folds in the "K" input box, and the number of repeats in the "Repeated K Fold" input box.
4. Select "OK" to run the Model Screening platform.
5. When the results dialog is displayed, click the red triangle menu.
6. Select "Save Results Table"
7. In JMP Pro's menu bar, select "Add-Ins => Confidence Curves".
8. Follow the displayed instructions to choose the results table generated in Step 6, the reference model, and the metric to be used for comparison.
9. Afterwards, a Graph Builder report should be generated similar to that of the one shown below: