Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Discussions
- :
- About finding Confusion Matrix for different Cutoff values in Logistic Regressio...

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Mar 1, 2015 5:40 AM
(11575 views)

How do I generate confusion metrics for using different cutoff values? For example, I want to find confusion matrix for cutoff = 0.3 or cutoff = 0.7? etc.

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Mar 1, 2015 11:52 AM
(18368 views)
| Posted in reply to message from sursangeet1 03/01/2015 08:40 AM

Hi Prakash,

You can generate a new confusion matrix by saving the logistic function, then setting up a column formula, and then using tabulate to make the table. I recorded a quick 2-min video showing the method I would use (it's not terribly complicated, but writing this out in steps started to seem more complicated than it is!).

I hope this helps!

edit Sept 19 2017: It looks like the video disappeared some time ago; my apologies! Here is the video again:

7 REPLIES 7

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Mar 1, 2015 11:52 AM
(18369 views)
| Posted in reply to message from sursangeet1 03/01/2015 08:40 AM

Hi Prakash,

You can generate a new confusion matrix by saving the logistic function, then setting up a column formula, and then using tabulate to make the table. I recorded a quick 2-min video showing the method I would use (it's not terribly complicated, but writing this out in steps started to seem more complicated than it is!).

I hope this helps!

edit Sept 19 2017: It looks like the video disappeared some time ago; my apologies! Here is the video again:

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Nice video!

Craige

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Would you mind sharing a link to the video referenced in the solution? Thank you!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

May 8, 2017 4:06 AM
(9454 views)
| Posted in reply to message from AaronNHorvitz 05/08/2017 03:29 AM

I believe this add-in (https://community.jmp.com/t5/JMP-Add-Ins/Alternate-Cut-off-Confusion-Matrix-Add-In/ta-p/22357) does what you are looking for. Be aware that the overall accuracy of the model is not likely to improve by changing the threshold probability - the main value is that when the costs of misclassification are asymmetric, changing the threshold can reduce the more costly type of errror.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Hi, would you please indicate where users can find this video?

I can't see any links to it on this Q&A page.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Sep 14, 2017 11:25 AM
(8168 views)
| Posted in reply to message from sursangeet1 03/01/2015 08:40 AM

Hi everyone,

I just re-uploaded the video to my original post. My apologies for it disappearing!

-julian

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Thanks Julian.

Also, I hope this helps. In JMP Pro, on the model output page, click arrow down in top left corner and select Confusion Matrix. Then open Set Probability Threshold in the appeared new section of the output. This will show a slider you can move to control Recall and Specificity. Unfortunately, it does not seem to calculate Precision and F1 score :( But, I like the dynamic slider for the table :)