turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Discussions
- :
- getting AIC/likelihood from model coefficients

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Dec 1, 2016 1:41 PM
(7093 views)

Maybe a bit off topic, but:

I'm getting a model from R GLMNet (a general regression elastic net package, because the customer won't upgrade from JMP 10) and I need to calculate the AIC. This package doesn't seem to give me likelihood. So ... is it possible to calculate the max likelihood from the coefficients? The model is made using the max likelihood so I'm hoping that there is some way to pull this information if I have the chosen model. It probably isn't, but I'm grasping at straws right now.

1 ACCEPTED SOLUTION

Accepted Solutions

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Dec 3, 2016 9:08 AM
(12057 views)
| Posted in reply to message from vince_faller 12/01/2016 04:41 PM

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

Learn it once, use it forever!

10 REPLIES

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

The R glmnet package manual provides these details:

"*A glmnet object has components dev.ratio and nulldev. The former is the fraction of (null) deviance explained. The deviance calculations incorporate weights if present in the model. The deviance is defined to be 2*(loglike_sat - loglike), where loglike_sat is the log-likelihood for the saturated model (a model with a free parameter per observation). Null deviance is defined to be 2*(loglike_sat -loglike(Null)); The NULL model refers to the intercept model, except for the Cox, where it is the 0 model. Hence dev.ratio=1-deviance/nulldev, and this deviance method returns (1-dev.ratio)*nulldev.*"

This information is intended to be used to assess goodness of fit.

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

So just so I'm making sure I have this correct.

deviance = (1-dev.ratio)*nulldev

and this deviance would be the exact same as the -2LogLikelihood in the AICc equation?

This could make my life MUCH better.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

I'm afraid that the two quantities available from the **glmnet** object (**dev.ratio**, **nulldev**) are not enough to obtain the likelihood for the model, which you need to compute AICc. You have two equations in three unknowns: likelihood(null), likelihood(model), and likelihood(saturated). I can't get the likelihood(model) free from the likelihood(null).

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Well thanks for trying either way.

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

The deviance is a measure of the lack of fit. It is not the likelihood of the model but twice the difference between likelihoods of the saturated model and your model.

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Dec 3, 2016 9:08 AM
(12058 views)
| Posted in reply to message from vince_faller 12/01/2016 04:41 PM

You might be able to get the likelihood of the model after all.

Fit the null model (no terms except the intercept) and save the glmnet object as fit0. Fit the desired model and save it as fit1. Get the deviance for both, that is dev0=deviance(fit0) and dev1=deviance(fit1). The deviance is 2(L(saturated)-L(model)), so now you have two equations and two unknown likelihoods. The L(saturated) is common and drops out. One of the components of the fit0 and fit1 objects if the null deviance (nulldev) to help.

So you were correct earlier. That is, you can use the deviance (somewhat) directly for computing AICc. Deviance(null) - Deviance(fit) = 2(L(model)-L(null)). The L(null) is just a constant offset so the difference in deviance is a substitute for 2L(model). Hope this helps!

BTW, the package provides the deviance ratio as a measure of goodness of fit. I know that is not the same as using AICc to select the best model. Rather like using R square (bad idea).

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Thanks for helping with this (On Saturday no less). Just so I'm clear, If I do the steps below, I should get something proportional to JMPs AIC?

```
Names Default to here(1);
k =[0, 1, 1, 2, 2];
n=500;
nulldev = 1100;
devs = [1100, 1099, 1098, 1097, 1096]; //calculated from (1-dev.ratio)*nulldev
fake2L = nulldev-devs;
AIC = -fake2L+2*k+2*k:*(k+1):/(n-k-1);
```

*Edit* changed AIC to be correct

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Dec 5, 2016 11:49 AM
(6826 views)
| Posted in reply to message from vince_faller 12/05/2016 02:45 PM

The first term in the equation should be negative. As the deviance decreases, AICc should get better (smaller).

Learn it once, use it forever!

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Excuse me, where is the reference stating that "Deviance(null) - Deviance(fit) = 2(L(model)-L(null))"?

Also, did you mean 2L(model) is 2 log likelihood? or the 2(L(model)-L(null) is the 2 log likelihood?