Random Effect vs Fixed Effects influence on Total model Rsq

[#1]

Hello everyone,

Had a question on how to interrupt the Rsq of a model with a random effect vs the % of contribution output. 

This is from the JMP sample datatable "Machine"

On the left is the model without the random factors and on the right is the output with them.  You can see that the Rsq goes from .53 to .99 when you add the random effects.

So the random effects can explain half the variation in the data so both random effects and fixed effects are very important.  However, the only way I know to be able to see that is by running both models.  Is there a way to look at the outputs of just the random effects model and get to the same conclusion?  I know you can visually get an idea of it by looking at the difference between the actual by predicted plots of the marginal and conditional models.

Thanks for any insight!!

Steve

[#2]

Hi @shampton82,

If you look at the equation of R², it involves a ratio between the sum of squares of residuals (from a model) and the total sum of squares (related to the variance of the data):

from Wikipedia : Coefficient of determination - Wikipedia

So R² is a metric indicating the proportion of variability explained by the model.

When part of the variance in the data is attributed to a random effect, this reduce the unexplained part of the variance, so it reduces the residual sum of squares, thus increasing R² (the same way a fixed effect is added).

You can also see this situation looking at RMSE improvement between the two models, going from 5,91 to 0,93 when random effects are included. If you change these effects from random to fixed effects, you'll have the same R² and RMSE metrics.

The inclusion of effects as random or fixed in a model is linked to the way the data has been collected/generated (for example by DoE), the role of the factors and hypothesis/assumptions about their influence on the response(s).

I will let other experts dive deeper into this topic.

Hope this first discussion starter will help,

[#3]

Thanks for the quick reply Victor!

I guess I am trying to see if there is a more quantitative way to know what amount of the Rsq the random effect is explaining vs the fixed effects.  Right now the only way I know to get this is to run both a mixed model and a fixed effect model and the difference between the two models Rsqs is how much the random effects are explaining.  Was hoping there was a way to determine this with just the mixed model outputs.

The reason I ask is that I've had models where the random effect explains almost all the variation in the data and I could see that in the actual by predicted plots but couldn't put a number to it.