Query regarding design using categorical factors

Manisha — Wed, 06 Aug 2025 12:34:20 GMT

JMP student edition 18
I was trying a custom design with 2 categorical factors and 3 levels. All effects -main and interaction show non significant p value despite using the recommended 9 run by the software.

Re: Query regarding design using categorical factors

Victor_G — Wed, 06 Aug 2025 23:32:46 GMT

Hi @Manisha !

Welcome in the Community !

What is your objective (s) with this DoE : identify important effects, optimize your system, test the robustness of your system against noise factors, ... ? What are your factors ? Why are they only categorical with 3 levels ?

There are many possibles explanations to why you may not find any effects statistically significant:

High noise/experimental variability in response / inadequate measurement precision & repeatability : Have you any replicate runs in your design that may help estimate experimental variability ?
Factors ranges too small : in your case you're using only categorical factors, but are the levels studied different enough to see some variation ?
Missing of other important factors in the design
Inappropriate model and/or factors definition
Maybe these factors are indeed statistically non-significant and impactful on the response studied and with the levels tested.
...

Regarding your design, I guess you have done all combinations involving your 2 three-levels factors (3x3). Maybe you could add some runs to have replicated runs and better estimate experimental variability, or adjust the p-value threshold depending on the power analysis : if you're very early in your study and regarding the small number of experiments, I doubt that a "standard" p-value threshold of 0.05 will enable to identify statistically significant effect (unless there are very strong effect sizes and very low noise/RMSE). You could use the Power analysis when designing your DoE with an estimate of anticipated noise in your response to adjust the size of the DoE and/or the alpha level (significance level).
What are the effects estimates ? Are they practically important ? Do they have a meaningful influence on the response ?

You can also read this similar topic (and responses) for information :
With more information about your context it will be easier to help you,

topic Query regarding design using categorical factors in Discussions

Query regarding design using categorical factors

Re: Query regarding design using categorical factors