cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
Not_A_Student
Level II

Stepwise regression: history plot. Why yellow band = min(BIC|AICc)+10 and green = min(BIC|AICc) + 5 ?

In both the stepwise regression and in the generalized regression there is a yellow and a green band in the history plot of the V (goodness of fit)  were v is either AICc or BIC.  

 

The instruction manual tells that yellow is  min(V) + 10 and green is min(V) + 5.

In the generalized regression platform it may also be used as a stopping rule like 'biggest inside yellow'.

 

But why the +10 and +5 and has there anyone justified those values in a monograph or a presentation or another referable source?

 

And why did JMP deliberately choose those 2 numbers?

 

Related pic:

Not_A_Student_0-1617812976357.pngNot_A_Student_1-1617813033646.png

 

 

Looking forward for feedback!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Stepwise regression: history plot. Why yellow band = min(BIC|AICc)+10 and green = min(BIC|AICc) + 5 ?

The documentation explains the use of the bands and cites the sources you want:

Comparable Model Zones

Although a model is estimated to be the best model, there can be uncertainty relative to this selection. Competing models might fit nearly as well and can contain useful information. For the AICc, BIC, KFold, and Leave-One-Out validation methods, and for a Validation Column with more than three values, the Validation Plot provides zones that identify competing models that might deserve consideration. Models that fall outside the zones are not recommended. See Burnham and Anderson (2004) and Burnham et al. (2011).

 

These criteria are not intended for inference. The information criteria are to be minimized, but like any metric, you should carefully examine the results and verify their validity, and not use them exclusively or blindly.

 

Please see the documentation for the Solution Path Plot and the associated references.

View solution in original post

1 REPLY 1

Re: Stepwise regression: history plot. Why yellow band = min(BIC|AICc)+10 and green = min(BIC|AICc) + 5 ?

The documentation explains the use of the bands and cites the sources you want:

Comparable Model Zones

Although a model is estimated to be the best model, there can be uncertainty relative to this selection. Competing models might fit nearly as well and can contain useful information. For the AICc, BIC, KFold, and Leave-One-Out validation methods, and for a Validation Column with more than three values, the Validation Plot provides zones that identify competing models that might deserve consideration. Models that fall outside the zones are not recommended. See Burnham and Anderson (2004) and Burnham et al. (2011).

 

These criteria are not intended for inference. The information criteria are to be minimized, but like any metric, you should carefully examine the results and verify their validity, and not use them exclusively or blindly.

 

Please see the documentation for the Solution Path Plot and the associated references.