Subscribe Bookmark RSS Feed

getting AIC/likelihood from model coefficients

vince_faller

Super User

Joined:

Mar 17, 2015

Maybe a bit off topic, but:

I'm getting a model from R GLMNet (a general regression elastic net package, because the customer won't upgrade from JMP 10) and I need to calculate the AIC.  This package doesn't seem to give me likelihood.  So ... is it possible to calculate the max likelihood from the coefficients? The model is made using the max likelihood so I'm hoping that there is some way to pull this information if I have the chosen model.  It probably isn't, but I'm grasping at straws right now.  

1 ACCEPTED SOLUTION

Accepted Solutions
markbailey

Staff

Joined:

Jun 23, 2011

Solution

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

Learn it once, use it forever!
8 REPLIES
markbailey

Staff

Joined:

Jun 23, 2011

The R glmnet package manual provides these details:

"A glmnet object has components dev.ratio and nulldev. The former is the fraction of (null) deviance explained. The deviance calculations incorporate weights if present in the model. The deviance is defined to be 2*(loglike_sat - loglike), where loglike_sat is the log-likelihood for the saturated model (a model with a free parameter per observation). Null deviance is defined to be 2*(loglike_sat -loglike(Null)); The NULL model refers to the intercept model, except for the Cox, where it is the 0 model. Hence dev.ratio=1-deviance/nulldev, and this deviance method returns (1-dev.ratio)*nulldev."

This information is intended to be used to assess goodness of fit.

Learn it once, use it forever!
vince_faller

Super User

Joined:

Mar 17, 2015

So just so I'm making sure I have this correct.  

deviance = (1-dev.ratio)*nulldev

and this deviance would be the exact same as the -2LogLikelihood in the AICc equation?

 

AICc.png

 

This could make my life MUCH better.  

 

markbailey

Staff

Joined:

Jun 23, 2011

I'm afraid that the two quantities available from the glmnet object (dev.ratio, nulldev) are not enough to obtain the likelihood for the model, which you need to compute AICc. You have two equations in three unknowns: likelihood(null), likelihood(model), and likelihood(saturated). I can't get the likelihood(model) free from the likelihood(null).

Learn it once, use it forever!
vince_faller

Super User

Joined:

Mar 17, 2015

Well thanks for trying either way.  

markbailey

Staff

Joined:

Jun 23, 2011

The deviance is a measure of the lack of fit. It is not the likelihood of the model but twice the difference between likelihoods of the saturated model and your model.

Learn it once, use it forever!
markbailey

Staff

Joined:

Jun 23, 2011

Solution

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

Learn it once, use it forever!
vince_faller

Super User

Joined:

Mar 17, 2015

Thanks for helping with this (On Saturday no less).  Just so I'm clear, If I do the steps below, I should get something proportional to JMPs AIC?

 

Names Default to here(1);

k =[0, 1, 1, 2, 2];
n=500;
nulldev = 1100;
devs = [1100, 1099, 1098, 1097, 1096]; //calculated from (1-dev.ratio)*nulldev
fake2L = nulldev-devs;
AIC = -fake2L+2*k+2*k:*(k+1):/(n-k-1);

*Edit* changed AIC to be correct 

markbailey

Staff

Joined:

Jun 23, 2011

The first term in the equation should be negative. As the deviance decreases, AICc should get better (smaller).

Learn it once, use it forever!