cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Submit your abstract to the call for content for Discovery Summit Americas by April 23. Selected abstracts will be presented at Discovery Summit, Oct. 21- 24.
Discovery is online this week, April 16 and 18. Join us for these exciting interactive sessions.
Choose Language Hide Translation Bar
Confidence curves

In the paper titled "Confidence curves: an alternative to null hypothesis significance testing for the comparison of clas...", Daniel Berrar details issues related to the use of null hypothesis significance testing for the comparison of classifiers and proposes the use of confidence curves as an alternative. After installing the add-in, you can generate confidence curves for comparing various classifiers with a reference classifier in the following manner:

 

  1. In JMP Pro's menu bar, select "Analyze => Predictive Modeling => Modeling Screening".
  2. Select appropriate parameters for the "Y, Response" and "X, Factor" fields.
  3. In the "Folded Crossvalidation" section of the dialog, enable the "K Fold Crossvalidation" option and set the number of folds in the "K" input box, and the number of repeats in the "Repeated K Fold" input box.
  4. Select "OK" to run the Model Screening platform.
  5. When the results dialog is displayed, click the red triangle menu.
  6. Select "Save Results Table"bfricke_1-1633369331409.png
  7. In JMP Pro's menu bar, select "Add-Ins => Confidence Curves".
  8. Follow the displayed instructions to choose the results table generated in Step 6, the reference model, and the metric to be used for comparison.
  9. Afterwards, a Graph Builder report should be generated similar to that of the one shown below:

bfricke_2-1633369726451.png