When conducting analysis of split-plot design experiments, if the Wald p-value for Whole Plots or Subplots is relatively large, should Whole Plots or Subplots be removed?

[#1]

When conducting analysis of split-plot design experiments, if the Wald p-value for Whole Plots or Subplots is relatively large, should Whole Plots or Subplots be removed?

[#2]

Hi @Rily_Maya,

What is your objectives with this DoE ?

What do the Whole plots/sub plots refer to physically ? 
Split-plots designs were introduced in agricultural context originally, where complete randomization was not feasible as treatments had to be applied into pre-determined fields parts. If you have chosen a split-plot design structure, that means at least one of your factor wasn't completely randomized, so you should stick with the Mixed model (and whole plot/subplot random effects) analysis.

Using split-plot design strategies enable to catch part of the variability and attribute it to a random factor (whole plot/subplot effects). The risk of removing your whole plot random effect is that the residual variance will increase (it will be containing the former residual variance + whole plot & subplot variances). So you might end up with different statistically significant effects, a different model, and some effects may be "hidden" by the "newly created" total residual variance.

Here is an example with Box Corrosion Split-Plot JMP dataset. The Whole plot effect is not significant like in your example (but still capture 90% of the total variance, whereas in your case whole plots+subplots random effects capture more than 70% of the total variance for OCV, and whole plot random effect capture more than 38% of the total variance for thickness) :

But take a look at p-values of the fixed effect tests.

Now when I'm relaunching the same model without whole plots random effects, the interaction effect and coating effect don't appear statistically significant (but furnace temp does) :

And the parameters estimation have higher standard errors for the hard-to-change factor (coating) and any interaction involving it (even if the parameter estimates have the same value) :

(Left: mixed model report / Right: Standard Least Squares model report)

So clearly, if you have designed your DoE/experimental setup with the whole plot/subplot situation, you should stay with it.

It's the most "proper" way to analyze your data with a model reflecting your experimental constraints (lack of full randomization).

You can also see my little presentation about Split-Plot designs here : Understanding Design of Experiments: Split-plot designs

Hope this answer will help you,

[#3 · Accepted solution]

Many of the questions Victor asked are necessary to understand your situation.  Split-plots are designs with restrictions on randomization.  Sometimes this is done for convenience (e.g., hard to change factors) or for efficiency when you want to reduce the cost of the experiment without losing much information.  In fact, there are times when a split-plot design is both more efficient and more effective (e.g., greater precision) than a randomized design.  See:

Box, G.E.P., Stephen Jones (1992), “Split-plot designs for robust product experimentation”, Journal of Applied Statistics, Vol. 19, No. 1

Box and Jones also show how to appropriately analyze such designs.  This may be a different interpretation than how Victor suggests the analysis proceed (The whole plot need not be a random effect!).