AUC Comparison Method

[#1]

Hello,

I am comparing two averaged models in the Model Comparison function in JMP Pro. When running the AUC Comparison function in JMP Pro found by clicking the red triangle in the Model Comparison output, what are the statistics used to generate the confidence intervals of the AUCs and the stats used to compare the two curves to generate a p value?

[#2]

The AUC is related to the non-parametric Mann-Whitney test statistic U based on ranks. See AUC section in ROC article.

[#3]

What specific method is utilized by JMP Pro to compare AUC's? Delong, Hanley & McNeil, or other?

[#4]

If you go to the ROC plot, and select the Helper Tool "?" and click in the window, it will bring up the help window for the chart.  There you will find that they reference 

DeLong, E. R., DeLong, D. M., and Clarke-Pearson, D. L. (1988). “Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach.” Biometrics 44:837–845.

[#5]

I am trying to do just this, compare two AUC.

I'm using JMP PRO 16.1.

I would appreciate if you could give a more details descriptions on how to do this.

TIA

[#6]

So, I tried using the Model comparison method (under Analyze --> Predictive Modeling). First I saved the probability formula of each of the models (using the red triangle in the model window). This added 4 columns to the data table for each model. In the window of the Model comparison I added the first formula column (since this is the linear predictor for computing the cumulative logits see here ) for each of the models and chose the appropriate column that it predicts.
Unfortunately, this resulted in the following message:
"Predictor column Model1_Lin[1] doesn't have property to identify which level it is predicting
Predictor column Model2_Lin[1] doesn't have property to identify which level it is predicting"
Obviously I'm doing something wrong.
Any tips would be appreciated.

[#7]

The column that you tried to use is only the estimated linear predictor for the logit. Use the one that computes the probability from the logit prediction.