cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
View Original Published Thread

About finding Confusion Matrix for different Cutoff values in Logistic Regression

sursangeet1
Level II

How do I generate confusion metrics for using different cutoff values? For example, I want to find confusion matrix for cutoff = 0.3 or cutoff = 0.7? etc.

1 ACCEPTED SOLUTION

Accepted Solutions
julian
Community Manager Community Manager


Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

Hi Prakash,

You can generate a new confusion matrix by saving the logistic function, then setting up a column formula, and then using tabulate to make the table. I recorded a quick 2-min video showing the method I would use (it's not terribly complicated, but writing this out in steps started to seem more complicated than it is!).

I hope this helps!

julian

 

edit Sept 19 2017: It looks like the video disappeared some time ago; my apologies! Here is the video again:

 

Different cutpoint for confusion matrix in Logistic Regression
Video Player is loading.
Current Time 0:00
Duration 3:30
Loaded: 4.69%
Stream Type LIVE
Remaining Time 3:30
 
1x
    • Chapters
    • descriptions off, selected
    • captions off, selected
    • en (Main), selected
    (view in My Videos)

     

    View solution in original post

    7 REPLIES 7
    julian
    Community Manager Community Manager


    Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

    Hi Prakash,

    You can generate a new confusion matrix by saving the logistic function, then setting up a column formula, and then using tabulate to make the table. I recorded a quick 2-min video showing the method I would use (it's not terribly complicated, but writing this out in steps started to seem more complicated than it is!).

    I hope this helps!

    julian

     

    edit Sept 19 2017: It looks like the video disappeared some time ago; my apologies! Here is the video again:

     

    Different cutpoint for confusion matrix in Logistic Regression
    Video Player is loading.
    Current Time 0:00
    Duration 3:30
    Loaded: 4.69%
    Stream Type LIVE
    Remaining Time 3:30
     
    1x
      • Chapters
      • descriptions off, selected
      • captions off, selected
      • en (Main), selected
      (view in My Videos)

       

      Craige_Hales
      Super User


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      Nice video!

      Craige
      AaronNHorvitz
      Level II


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      Would you mind sharing a link to the video referenced in the solution?  Thank you! 

       

       

      dale_lehman
      Level VII


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      I believe this add-in (https://community.jmp.com/t5/JMP-Add-Ins/Alternate-Cut-off-Confusion-Matrix-Add-In/ta-p/22357) does what you are looking for.  Be aware that the overall accuracy of the model is not likely to improve by changing the threshold probability - the main value is that when the costs of misclassification are asymmetric, changing the threshold can reduce the more costly type of errror.

      Oleg
      Level III


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      Hi, would you please indicate where users can find this video?
      I can't see any links to it on this Q&A page.

      julian
      Community Manager Community Manager


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      Hi everyone,

       

      I just re-uploaded the video to my original post. My apologies for it disappearing!

      -julian

       

       

      Oleg
      Level III


      Re: About finding Confusion Matrix for different Cutoff values in Logistic Regression

      Thanks Julian. 

      Also, I hope this helps. In JMP Pro, on the model output page, click arrow down in top left corner and select Confusion Matrix. Then open Set Probability Threshold in the appeared new section of the output. This will show a slider you can move to control Recall and Specificity. Unfortunately, it does not seem to calculate Precision and F1 score :( But, I like the dynamic slider for the table :)

       

      Screen Shot 09-28-17 at 11.06 AM.PNG Screen Shot 09-28-17 at 11.07 AM.PNG