Design of Experiments Club Discussions

maria_astals

Video 1: Introduction to JMP 19

Video 2: Including covariates and/or uncontrolled supplementary variables in DOE set-up and analysis.

Video 3: Calculating power of your sample size in DOE, particularly for in vivo (i.e., animal studies) where replicates are common and necessary.

(view in My Videos)

Video 4: How to choose the appropriate design for a DoE experiment?

(view in My Videos)

Q3 DOE Club Q&A compiled

How should covariates and uncontrolled supplementary variables be handled in the setup and analysis of a DOE?

Designed Experiments often face variables outside the planned factors called covariates. Some are measurable but not controlled (covariates, e.g., baseline weight, pH), while others are uncontrolled noise (e.g., humidity, operator effects). Ignoring these can inflate error and obscure true factor effects.

DOE Approach:

Covariates: If measurable, add them into the model. This reduces unexplained variation, sharpens estimates, and increases power.
Uncontrolled supplementary variables: If not measurable, manage through blocking (grouping similar conditions), randomization (spreading nuisance effects evenly), and replication (estimating variability).

Solution:
Identify likely covariates during the Define stage, measure them consistently, and include them in the model. For unmeasurable noise factors, rely on blocking, randomization, and replication. This ensures cleaner signal detection and reliable conclusions, especially in complex biological or manufacturing DOE.

Reference:

https://community.jmp.com/t5/JMP-Blog/What-is-a-covariate-in-design-of-experiments/ba-p/361517

https://community.jmp.com/t5/Discussions/DOE-with-covariate/td-p/825007

https://community.jmp.com/t5/Discussions/Unexpected-change-in-controlled-variable-in-DOE/td-p/729169

https://community.jmp.com/t5/Discussions/Mixture-DOE-with-covariate-batch-effects/td-p/718593

https://community.jmp.com/t5/Learn-JMP-Events/Developer-Tutorial-Handling-Covariates-Effectively-whe...

https://community.jmp.com/t5/JMPer-Cable/New-in-JMP-16-Improved-covariate-handling-in-DOE/ba-p/36154...

https://community.jmp.com/t5/Discussions/How-to-Account-for-Uncontrolled-Factors-Using-a-Definitive/...

How can we calculate the power of a DOE sample size, particularly in in vivo (animal) studies where replicates are common and necessary?

In in vivo studies, sample size is limited by cost, ethics, and feasibility. Too few animals lead to low power (risk of missing real effects), while too many waste resources and may raise ethical concerns.

DOE Approach:

Power is the probability of detecting a true effect given sample size, effect size, variability, and significance level.
Replicates are especially important in animal studies because biological variability is high.
JMP’s Power and Sample Size platform lets you simulate different scenarios before running the study.

Solution:
Estimate expected variability from pilot data or literature. Use power analysis to determine the minimum number of replicates needed to achieve acceptable power (commonly 80%). If resource constraints limit sample size, design the DOE to focus on the most critical factors, and consider blocking or covariates to reduce noise. This balances statistical rigor with ethical and budget realities.

Reference:

https://community.jmp.com/t5/Discussions/How-to-use-sample-size-and-power-for-factorial-DoE-to-deter...

https://community.jmp.com/t5/Discussions/Power-calculations-to-determine-sample-and-replication-of/t...

https://community.jmp.com/t5/Discussions/Sample-Size-for-DOE/td-p/237085

https://community.jmp.com/t5/Discussions/Power-Equation/td-p/53219

https://community.jmp.com/t5/Discussions/Should-I-consider-power-analysis-in-DOE/td-p/501063

https://community.jmp.com/t5/Discussions/DoE-How-to-treat-replicate-measurements/td-p/18588

https://community.jmp.com/t5/Discussions/Power-Analysis-in-Design-Evaluation-DOE/td-p/65472

https://community.jmp.com/t5/Discussions/Doe-and-replications/td-p/565281

https://community.jmp.com/t5/Learning-Center/Sample-Size-Explorers-DOE/ta-p/414885

https://community.jmp.com/t5/Discussions/quot-Animals-quot-Split-Plot-DOE/td-p/546140

When drafting a DOE exploration path, how do we determine the number of experiments required to achieve specific goals (such as coverage of design space or reduction of uncertainty), and how can this be balanced against resource constraints?

When planning a DOE, there is always tension between the ideal number of runs (to cover the design space and reduce uncertainty) and the maximum feasible runs due to time, cost, or ethical limits. Without careful planning, you risk either underpowered results or wasted resources.

DOE Approach:

Use design diagnostics (like prediction variance profiles, design space coverage, and aliasing structures) to quantify how well a candidate design explores the factor space.
Apply power analysis to check if the chosen sample size can detect effects of practical importance.
Explore sequential/iterative designs (like Bayes Opt) starting smaller and expanding only if needed, instead of committing all resources upfront.

Solution:
Define the experimental goals clearly (effect detection, response surface mapping, or robustness testing). Then use DOE tools to evaluate candidate designs against those goals under the given resource ceiling. If the maximum allowable experiments do not provide sufficient coverage or power, either

narrow the study scope to critical factors and responses, or
adopt a staged design strategy. This ensures that even with constraints, the DOE produces meaningful, defensible insights.

https://community.jmp.com/t5/Discussions/how-JMP-design-the-number-of-runs-for-Default-under-DOE-cus...

https://community.jmp.com/t5/Discussions/JMP-Custom-DOE-vs-Full-Factorial-DOE/td-p/256413

https://www.jmp.com/content/dam/jmp/documents/en/support/jmp111/DOE_Guide.pdf

https://community.jmp.com/t5/Learn-JMP-Events/Understanding-DOE-Capabilities-in-JMP/ev-p/809981

What is the best way to choose the appropriate experimental design for a DOE study?

Selecting an inappropriate design risk wasting resources and failing to answer the study’s questions.

DOE Approach:

Match design to objective:
Screening : Fractional Factorial, Definitive Screening Design (DSD).
Optimization/curvature modeling : Response Surface Designs (Central Composite, Box–Behnken).
Mixture/formulations: Simplex lattice, Simplex centroid, or Mixture Space-Filling designs.
Robustness/noise: Taguchi or robust designs.
Custom Design in JMP: When standard catalog designs don’t fit (due to unusual factor types, constraints, or resource limits), JMP’s Custom Design platform builds an experiment tailored to your factors, model, and number of runs.
Always check diagnostics (alias structure, prediction variance, design efficiency) to validate the choice.

Solution:
Define the experimental goal clearly (screening, optimization, robustness, mixtures). Use classic designs where appropriate, but rely on JMP’s Custom Design to handle complex, real-world scenarios. This ensures flexibility, efficient use of resources, and alignment with the actual scientific question.

https://www.synthace.com/blog/types-of-doe-design-a-users-guide

https://ilssi.org/how-to-choose-the-right-design-for-a-designed-experiment-doe/

https://community.jmp.com/t5/Abstracts/Design-of-Experiment-s-Crucial-Step-0-Choosing-the-Right-DOE/...

https://www.jmp.com/content/dam/jmp/documents/en/support/jmp111/DOE_Guide.pdf

https://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm