topic Re: getting AIC/likelihood from model coefficients in Discussions

getting AIC/likelihood from model coefficients

vince_faller — Thu, 01 Dec 2016 21:41:28 GMT

Maybe a bit off topic, but:

I'm getting a model from R GLMNet (a general regression elastic net package, because the customer won't upgrade from JMP 10) and I need to calculate the AIC. This package doesn't seem to give me likelihood. So ... is it possible to calculate the max likelihood from the coefficients? The model is made using the max likelihood so I'm hoping that there is some way to pull this information if I have the chosen model. It probably isn't, but I'm grasping at straws right now.

Re: getting AIC/likelihood from model coefficients

Mark_Bailey — Fri, 02 Dec 2016 13:29:19 GMT

The R glmnet package manual provides these details:

"A glmnet object has components dev.ratio and nulldev. The former is the fraction of (null) deviance explained. The deviance calculations incorporate weights if present in the model. The deviance is defined to be 2*(loglike_sat - loglike), where loglike_sat is the log-likelihood for the saturated model (a model with a free parameter per observation). Null deviance is defined to be 2*(loglike_sat -loglike(Null)); The NULL model refers to the intercept model, except for the Cox, where it is the 0 model. Hence dev.ratio=1-deviance/nulldev, and this deviance method returns (1-dev.ratio)*nulldev."

This information is intended to be used to assess goodness of fit.

Re: getting AIC/likelihood from model coefficients

vince_faller — Fri, 02 Dec 2016 20:43:05 GMT

So just so I'm making sure I have this correct.

deviance = (1-dev.ratio)*nulldev

and this deviance would be the exact same as the -2LogLikelihood in the AICc equation?

This could make my life MUCH better.

Re: getting AIC/likelihood from model coefficients

Mark_Bailey — Fri, 02 Dec 2016 21:56:21 GMT

I'm afraid that the two quantities available from the glmnet object (dev.ratio, nulldev) are not enough to obtain the likelihood for the model, which you need to compute AICc. You have two equations in three unknowns: likelihood(null), likelihood(model), and likelihood(saturated). I can't get the likelihood(model) free from the likelihood(null).

Re: getting AIC/likelihood from model coefficients

Mark_Bailey — Fri, 02 Dec 2016 21:58:10 GMT

The deviance is a measure of the lack of fit. It is not the likelihood of the model but twice the difference between likelihoods of the saturated model and your model.

Re: getting AIC/likelihood from model coefficients

vince_faller — Fri, 02 Dec 2016 21:58:25 GMT

Well thanks for trying either way.

Re: getting AIC/likelihood from model coefficients

Mark_Bailey — Mon, 05 Dec 2016 14:12:42 GMT

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

Re: getting AIC/likelihood from model coefficients

vince_faller — Mon, 05 Dec 2016 20:40:33 GMT

Thanks for helping with this (On Saturday no less). Just so I'm clear, If I do the steps below, I should get something proportional to JMPs AIC?

Names Default to here(1);

k =[0, 1, 1, 2, 2];
n=500;
nulldev = 1100;
devs = [1100, 1099, 1098, 1097, 1096]; //calculated from (1-dev.ratio)*nulldev
fake2L = nulldev-devs;
AIC = -fake2L+2*k+2*k:*(k+1):/(n-k-1);

*Edit* changed AIC to be correct

Re: getting AIC/likelihood from model coefficients

Mark_Bailey — Mon, 05 Dec 2016 19:49:47 GMT

The first term in the equation should be negative. As the deviance decreases, AICc should get better (smaller).

Re: getting AIC/likelihood from model coefficients

hafidwr — Tue, 21 Nov 2017 05:56:52 GMT

Excuse me, where is the reference stating that "Deviance(null) - Deviance(fit) = 2(L(model)-L(null))"?

Also, did you mean 2L(model) is 2 log likelihood? or the 2(L(model)-L(null) is the 2 log likelihood?

Re: getting AIC/likelihood from model coefficients

hafidwr — Tue, 21 Nov 2017 06:20:35 GMT

Excuse me, do you have reference on "Deviance(null) - Deviance(fit) = 2(L(model)-L(null))"?

With 2(L(model)-L(null) do you mean it's "2 log likelihood" or "2 log (L0/L1)"? Shouldn't it be "-2 log likelihood", or the equation of Deviance(null) - Deviance(fit) ignore the negative (-) sign?

Sorry for the rapid question.