Negative R2 in Mixed Models

[#1]

How is R2 calculated in JMP (v12.2) when fixed and random effects are included in Fit Model? I can't find any documentation on this. The reason this came up is that I have been getting negative R2 values.   

[#2]

Have you tried using the "?" tool and clicking on the Results Table you are interested in finding the details on?

To do this, either select the "? Help" tool from the Tools pull down menu and then click on the table you are interested in, or input Shift/? to bring the tool up, and then click on the table of interest.

It is a good start on finding out the details.

[#3]

I have tried the "?" tool (which usually gets me what I want), but it takes me to the summary of fit for fixed effect models only (no random effects) which gives the standard definition, R2 =SSM/SST. But I don't think this applies for mixed models and if it does there is no ANOVA report to get the SSM and SST.

[#4]

Hi @jwiltsie,

Here is a section of the JMP help that might be of use: 

http://www.jmp.com/support/help/Mixed_and_Random_Effect_Model_Reports_and_Option.shtml

Without seeing your particular results I can't be sure what is happening, but I suspect the model you're fitting has a number of random effects that are likely close to zero/absorbing no variance. When fitting a mixed model, you have the option to allow "unbounded variance components," which means that a variance component can converge to a negative estimate. This might seem strange (or downright ridiculous) since we know in reality a variance can never be negative. But, it's been shown that constraining estimates for the variance components to be positive leads to bias in estimation of the fixed effects (if you're interested in the statistical reasons for this I've included a google scholar search below). 

So, what does that have to do with your coefficient of determination for the full model? If many of your variance components are estimated to be negative (meaning they're likely inactive in the population), and whatever fixed effects you're estimating absorb little variance, it's possible for the overall model R2 to be negative. 

As I said before, it's hard to know what's happening exactly without seeing more about your particular situation, but hopefully this helps put you on the right path!

- Julian

https://scholar.google.com/scholar?q=unbounded+variance+components+mixed+model&hl=en&as_sdt=0&as_vis...

[#5]

Thanks @julian,

Your description is correct in that I have a random effect with a negative estimate, here is a screen shot (should have included this in the original post).

Mixed Model Output with negative R2

I was wondering about the statistical details for R2 for mixed models (which I can't find in the link you provided). Based on a quick internet search it looks like there is more than one method to calculate R2 for mixed models.

[#6]

Hi @jwiltsie,

Sorry, I clearly didn't answer the question you asked :) I just looked through the documentation again and although there are statistical details for R^2 for fixed-effects only models, I also can't seem to find details for models with random effects. I will do some more digging, but I would suggest you email support@jmp.com with your question and they will likely be able to get the answer to your question quickly. 

Julian

[#7]

Is it reporting the Generalized RSquare as described in the Mixed Model Personality (JMP Pro only, which I don't have)? This excerpt is from the JMP Fitting Linear Models Book under the Help menu. It looks like this would be reasonable although I don't get a negative R2 when I try this calculation: