Using the custom designer a 36 run DOE was created for 7 factors: 5 continuous and 2 categorical. Each day 4 runs are performed so a blocking factor with 9 blocks is added to the input parameter list. DOE evaluation shows a nice parameter power, acceptable low correlation among the factors and the blocks nearly orhogonal to the factors. Analysis of the results by fitting a response surface can be done in several ways; which one of below is correct/recommended? I consider block as a random effect.
1. Transforming block to random effect and using Standard least squares + REML: after removing non effective fparameters --> 17 active effects, lot of interactions & quadratic effects: R² = 0,97 R²adj = 0,94 AICc = 327 This looks to overfitting to me..? However R²adjusted is still close to R²...
2. Stepwise & block = fixed effect (stepwise does not accept random effects?):10 fixed effects: 3 fixed block effects and 7 parameters --> 5 main effects, 1 interaction and 1 quadratic effect. Consideringt block as a random effect, before making the model I transformed the 3 fixed block effects to 3 random effects, is this correct? Making the REML model I get R² = 0,90 R²adj = 0,87 AICc = 256
The two models are clearly different! I would prefer the second model with lower #effects & AICc and still R² = 0,9. There are too few runs to create a testset so what is your opinion?
Remark: in the 2nd, stepwise procedure, instead or assigning the 3 fixed block effects as 3 random effects I also can create a REML model by taking up the 9 level block as one random effect, is this the right way?
Thanks for input! Frank
OK thanks for inputs. Pity that stewise can't handle random effects, would be great; backward analysis is cumbersome when many factors are involved. Is the AICc criterion not OK for judging the models; the first model has 17 significant effect R² = 0,98 but AICc is much higher than the 2nd lower #effect model with R² = 0,9 ?
Thanks Cameron for this useful comment; I will have a look at this more or less forgotten DOE screening analysis platform. I agree that the fixed block approach consumes lot of degrees of freedom, on the other hand this will make sure that finally only strong effects will be screened out so I am happy with my lower but stonger effect R² = 0,9 model; I have tried the cumbersome standard least squares backward selection and sometimes it is hard to judge when to remove an effect or not; would you reject an interaction effect with p = 0,075? Using backward selection, nearly every time I find another model.. judgement of p is critical!! I prefer fixed effect/stepwise and indeed by creating the final model block must be transformed to a random effect. Regards, Frank
The screening platform is for two-level factors. It will model any factors with more than two levels (such as 9 blocks) as powers of fixed effects, up to the 8th power in this case, and not as random effects. You should not use the Screening platform in this case.
Also, this simple platform is for screening factors, not effects. It ignores your specified model to create contrasts based on the key principles of screening.