Choose Language Hide Translation Bar
Highlighted
Lu
Lu
Level III

confusion matrix and performance meassures

At present in JMPpro, the only performance meassure automatic calculated from the confusion matrix from a model is the misclassification rate and eventually the AUC of the ROC. Other meassures such as Fase positive and  False negative rates, sensitivity , specificity, negative and positive predictive value, precision/recall (= F1-score) and MCC are commonly described in ML literature and are very usefull for model comparison. Online calculators exist but need manual data entry and are sensitive to data entry errors (ref: website for formulas http://onlineconfusionmatrix.com/). I have no experience  with the development of Add-Inns. An Add-Inn which calculates these peformance meassures from the confusion matrix would be very welcome for the JMP user searching for the optimal ML model. Indeed, ROC and  misclassification rate is far away from the best performance meassure especially when data are highly inbalanced, a very common situation in medical classification. My question, is it usefull to develop this Add-Inn and is there somebody on the forum who is able to put some effort in it?

Regards

 

Lu

1 REPLY 1
Highlighted
KarenC
Super User

Re: confusion matrix and performance meassures

Hi Lu,

Check out these two add-ins to see if they help with what you are looking for. The second one has much of what you want but does tend to run slowly so be patient. The first one I created specifically to address measures I wanted for diagnostic tests.

 

performance summary add-in 

 

model classification threshold explorer 

 

Article Labels