cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Try the Materials Informatics Toolkit, which is designed to easily handle SMILES data. This and other helpful add-ins are available in the JMP® Marketplace
Choose Language Hide Translation Bar
View Original Published Thread

getting AIC/likelihood from model coefficients

vince_faller
Super User (Alumni)

Maybe a bit off topic, but:

I'm getting a model from R GLMNet (a general regression elastic net package, because the customer won't upgrade from JMP 10) and I need to calculate the AIC.  This package doesn't seem to give me likelihood.  So ... is it possible to calculate the max likelihood from the coefficients? The model is made using the max likelihood so I'm hoping that there is some way to pull this information if I have the chosen model.  It probably isn't, but I'm grasping at straws right now.  

Vince Faller - Predictum
1 ACCEPTED SOLUTION

Accepted Solutions


Re: getting AIC/likelihood from model coefficients

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

View solution in original post

10 REPLIES 10


Re: getting AIC/likelihood from model coefficients

The R glmnet package manual provides these details:

"A glmnet object has components dev.ratio and nulldev. The former is the fraction of (null) deviance explained. The deviance calculations incorporate weights if present in the model. The deviance is defined to be 2*(loglike_sat - loglike), where loglike_sat is the log-likelihood for the saturated model (a model with a free parameter per observation). Null deviance is defined to be 2*(loglike_sat -loglike(Null)); The NULL model refers to the intercept model, except for the Cox, where it is the 0 model. Hence dev.ratio=1-deviance/nulldev, and this deviance method returns (1-dev.ratio)*nulldev."

This information is intended to be used to assess goodness of fit.

vince_faller
Super User (Alumni)


Re: getting AIC/likelihood from model coefficients

So just so I'm making sure I have this correct.  

deviance = (1-dev.ratio)*nulldev

and this deviance would be the exact same as the -2LogLikelihood in the AICc equation?

 

AICc.png

 

This could make my life MUCH better.  

 

Vince Faller - Predictum


Re: getting AIC/likelihood from model coefficients

I'm afraid that the two quantities available from the glmnet object (dev.ratio, nulldev) are not enough to obtain the likelihood for the model, which you need to compute AICc. You have two equations in three unknowns: likelihood(null), likelihood(model), and likelihood(saturated). I can't get the likelihood(model) free from the likelihood(null).

vince_faller
Super User (Alumni)


Re: getting AIC/likelihood from model coefficients

Well thanks for trying either way.  

Vince Faller - Predictum


Re: getting AIC/likelihood from model coefficients

The deviance is a measure of the lack of fit. It is not the likelihood of the model but twice the difference between likelihoods of the saturated model and your model.


Re: getting AIC/likelihood from model coefficients

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

vince_faller
Super User (Alumni)


Re: getting AIC/likelihood from model coefficients

Thanks for helping with this (On Saturday no less).  Just so I'm clear, If I do the steps below, I should get something proportional to JMPs AIC?

 

Names Default to here(1);

k =[0, 1, 1, 2, 2];
n=500;
nulldev = 1100;
devs = [1100, 1099, 1098, 1097, 1096]; //calculated from (1-dev.ratio)*nulldev
fake2L = nulldev-devs;
AIC = -fake2L+2*k+2*k:*(k+1):/(n-k-1);

*Edit* changed AIC to be correct 

Vince Faller - Predictum


Re: getting AIC/likelihood from model coefficients

The first term in the equation should be negative. As the deviance decreases, AICc should get better (smaller).

hafidwr
Level I


Re: getting AIC/likelihood from model coefficients

Excuse me, where is the reference stating that "Deviance(null) - Deviance(fit) = 2(L(model)-L(null))"? 

Also, did you mean 2L(model) is 2 log likelihood? or the 2(L(model)-L(null) is the 2 log likelihood?