Hi @dale_lehman ,
I can confirm that the alternative cut-off confusion matrix add-in doesn't work on the prediction columns you mentioned in your post. However, I re-ran the Boosted Tree platform following the same predictors and response that you used, even verified the equation was the same, but the alternative cut-off matrix add-in worked correctly on this new formula. I found the same thing with the fit nominal logistic. If I re-run it (I do end up with a different equation), I can run the alternative cut-off and it works as expected. I'm running JMP Pro 15.2.1. Something might be a bit off with your two fits. Have you checked the ROC curves for the fits to see if they're above the diagonal line. There was no saved script for those two model formulas in your table, so it's hard to assess how well the fits work. Have you tried re-fitting the logistic and boosted tree models again to see if the newer equations work? When I run them a second time, it works with the add-in.
On another note, I noticed that you used a random seed in generating your validation column. Since you have very few instances of stroke==1, you might consider making a validation column that is stratified on stroke so that it can take this into account. Also, when performing the nominal logistic fit, it might be better to model the Target Level: 1 rather than 0, since you're probably trying to predict the likelihood of a stroke (stroke==1) versus the likelihood of no stroke (stroke==0).
Just some thoughts. Hope this helps.
Good luck!,
DS