Main effect screening design vs. OFAT: which is best?

[#1]

Hello,

In JMP, under DOE > Classical > Two Level Screening > Screening Design > "Construct a main effects screening design", there is the possibility to create a screening design investigating main effects only. The result with 8 parameters is shown in attachment.

What would be the advantage of this design compared to an OFAT, where each of the 8 parameters would be investigated at a high and low level (2 runs per parameter)?

In both cases the number of runs is 16.

Thank you in advance for your support!

Best regards,

Claire

[#2]

Hi @cbaril,

The question of this post is a bit disturbing, as the answer would definitely be DoE, so I would recommend you to watch some webinars about the basics of DoE to better assess the relevance and importance of DoEs.

DoE has many benefits compared to OFAT, here are some :

• Limited (and planned/controlled in advance !) number of experiments,
• Interactions and non-linear effects can be studied,
• Mathematical model to explain or predict response(s) variations,
• High efficiency (information vs. number of experiments) and flexibility of the assumed model,
• Errors are shared equally/homogeneously in the design space,
• Possibility to create constraints/customs designs,
• Iterative process through "Augmentation" and a lot of strategies possible : you can start by screening main effects, then checking interactions, and finally build a robust Response Surface Model for predicting accurately your responses, ... 
• Randomization of the runs avoid influence of noise and/or non-controlled variables (which you wouldn't have with an OFAT approach).

You can also answer your question by creating an OFAT design and compare it to a fractional factorial design (with 16 runs, you would have the possibility in the case of the DOE to fit some 2-factors interactions, compared to the OFAT "screening") :

• Power analysis : Much higher possibility to detect significant main effects compared to OFAT
  
• Prediction variance over the experimental space : Better repartition of the runs in the experimental space, so lower prediction variance
  
• Correlations/aliases between terms : With OFAT design, no possibility to detect interactions (aliases)
  

You can reproduce the design evaluation and comparison with the datatables in copy.

You may find some ressources and further informations in these posts : 

Solved: DOE approach - JMP User Community

Solved: using DOE for OFAT experiments - JMP User Community

You can also join the Design of Experiments Club to exchange and learn more about DOE.

I hope this answer will help you,

[#3]

Hello Victor,

Thank you for your detailed reply and sorry for the disturbance! 

I did not want to compare DoE vs. OFAT in general but was wondering, for the specific case where someone would want to screen only for main effects, if a DoE would have an advantage over an OFAT. In this case it would be necessary to challenge the request and explain that there is a high likelihood of having interaction effects. Hence the need, in any case, to use a DoE. Correct?

What still confuses me is that there is a DoE design in JMP called "Construct a main effects screening design", so I thought it exclusively studied main effects, which an OFAT can do as well. The model terms that appear as default in the Model specification dialogue of the "Construct a main effects screening design" design include indeed only main effects. Is this specific design then no use at all?

Best regards,

Claire

[#4 · Accepted solution]

Hi @cbaril,

Even if you only want to screen main effects (and not even look at interactions), DoE is a better and more efficient methodology compared to OFAT, as you will have higher power (probability of detecting a significant effect if active), lower prediction variance, and better correlation structure between your inputs (you can see and compare the designs from the datatables sent earlier).

For 8 factors, you would need a minimum of 9 runs to estimate main effects coefficients terms with a DoE, compared to 16 with OFAT (as you would create a pair of experiments for each factor, with -1 and 1, and the other factors at 0 or other fixed levels).

In your example of 16 runs-DoE, there are more run than the minimum required, so there are some extra runs that helps increasing the power and precision of the main effect coefficients in the regression model, and also offer the possibility to include some 2-factors interactions (which is impossible to do with an OFAT approach).

So even if you want to screen main effects only, DoE is a more reliable, precise and informative approach than OFAT.

[#5]

Victor has already provided much detail, so I'll keep my thoughts short. An important feature of experimentation vs. OFAT that I want to reiterate.  OFAT's have an extremely narrow inference space.  Conclusions from OFAT's are contingent on every other factor that was held constant being at those constant settings in the future.  This is most likely completely unrealistic.  The 8 factors in the DOE has a much wider inference space (the design space is 8 dimensional).  OFAT's are 1 dimensional. Even though you may prioritize main effects in an experiment, the higher order effects are still present, they are just confounded.