confusion matrix and performance meassures

Lu — Fri, 09 Jun 2023 00:20:13 GMT

At present in JMPpro, the only performance meassure automatic calculated from the confusion matrix from a model is the misclassification rate and eventually the AUC of the ROC. Other meassures such as Fase positive and False negative rates, sensitivity , specificity, negative and positive predictive value, precision/recall (= F1-score) and MCC are commonly described in ML literature and are very usefull for model comparison. Online calculators exist but need manual data entry and are sensitive to data entry errors (ref: website for formulas http://onlineconfusionmatrix.com/). I have no experience with the development of Add-Inns. An Add-Inn which calculates these peformance meassures from the confusion matrix would be very welcome for the JMP user searching for the optimal ML model. Indeed, ROC and misclassification rate is far away from the best performance meassure especially when data are highly inbalanced, a very common situation in medical classification. My question, is it usefull to develop this Add-Inn and is there somebody on the forum who is able to put some effort in it?

Regards

Re: confusion matrix and performance meassures

KarenC — Sat, 22 Aug 2020 16:55:13 GMT

Hi Lu,

Check out these two add-ins to see if they help with what you are looking for. The second one has much of what you want but does tend to run slowly so be patient. The first one I created specifically to address measures I wanted for diagnostic tests.

performance summary add-in

model classification threshold explorer

topic Re: confusion matrix and performance meassures in Discussions

confusion matrix and performance meassures

Re: confusion matrix and performance meassures