Stepwise model selection for mixed model

[#1]

Dear all,

I want to use stepwise model selection in order to select important fixed effects from a mixel model (including one random effect).

When changing the "Personality" in the Fit Model Platform from "Standard Least Squares" to "Stepwise" it seems, however, that the same model selection is performed as if I would have a "pure" fixed effects model, not taking into account the random effect. Is there a way to take the random effect into account and then perform the stepwise model selection? I mean, if the random effect causes a large part of the overall variation in the data, this should make a difference, right?

Best regards,

Tina

[#2 · Accepted solution]

Hi @Tina,

That's always been a tricky situation in JMP since random effects are not supported in Stepwise.  If your random effect can temporarily be treated as fixed, you could run stepwise to get to a model and then revert to a random effect at that point.

I usually do a manual model selection in JMP starting with a full model and carefully paring it down.  You could always do it that way as well.

Best of luck.

[#3]

Hi, Tina!

Hopefully, you are aware of the fatal statistical flaws of Stepwise.  Please reference the attached document from NE SUG 2007.

A quote from that document: "Most devastatingly, it allows the analyst not to think. Put in another way, for a data analyst to use stepwise methods is equivalent to telling his or her boss that his or her salary should be cut."

I recommend you eschew Stepwise and try the Generalized Regression personality in the Fit Model platform instead, specifically the Adaptive Lasso.

Good luck!

[#4]

FYI, the methods @Kevin_Anderson suggests are only available in JMP Pro.

[#5]

Hi @cwillden and @Kevin_Anderson, thank you very much for your thoughts on that topic and tips! 

As the analysis has to run in "normal" JMP (no Pro version) I will try to pursue the way @cwillden proposed.

As I always want to have a model containing all main effects and selecting relevant interactions or quadratic effects in addition, I will start from the main effects model and put the random effect as fixed effect during stepwise.

Best,

Tina