Model Selection for Designed Experiments with Blocks

[#1]

I sometimes get questions about how to select the "best" model from experiments with blocks. We recommend that you model blocks as random effects, which means that you can't use Stepwise or Generalized Regression personalities for model selection because they only support fixed effects.

In this video I talk about:

• Why we block experiments
• Fixed versus random effects
• REML modelling
• Model selection strategies with random effects

I also mention the book, Optimal Design of Experiments: A Case Study Approach, as a good resource to learn more and the free chapter download that is available.

The example data table can be found from the attached JMP journal.

Let me know in the comments, below, if you have any questions.

Model Selection for Experiments with Blocks.jrn

[mjz5448]

When you fit the block as a fixed effect and used the stepwise personality to reduce the model, if the block had not come up as significant then is there any reason to add it back into the model as a random effect, or can you just leave it out all together

[Phil_Kay]

HI @mjz5448 ,

The recommendation is to keep the block effect in. This is because it will affect the statistical significance tests on your factor effects. There is error that is properly attributed to the block  and you don't want this to "contaminate" the error that is estimated for the factor effects. Hope that makes sense. In most cases it will not make a big difference to your conclusions.

Phil

[mjz5448]

Thanks @Phil_Kay , 

So are you saying the reason for leaving a non-significant block effect in a model as opposed to a non-significant factor effect is due to the fact that you're more certain (or maybe 100% certain?) that you can attribute a certain amount of error to the block, where as with main effects or higher order terms it become more ambiguous due to confounding effects etc..? 

[Phil_Kay]

@mjz5448  - sort of. We know that there is a source of error due to changes between blocks. So we should include the block effect to capture this source of error, even if it is not found to be statistically significant. This will mean that we have the best estimate of error for significance testing of our factor effects. Another very good reason for including the block effect is because @Mark_Bailey says that you should. That is always a good enough reason for me!