How to Evaluate DOE Design

[#1]

Hi all,

I’m working on a DOE with the following factors:

• 1 categorical variable with 4 levels

• 2 continuous variables 

Before running the experiments, I want to know how to evaluate my design to ensure that I can:

• Confidently identify which variables have the most influence on the response

• Determine which specific levels of the variables perform better

Also, I don’t want to do just the minimal number of runs, since I plan to run each experiment in replicate. Any advice on how to assess the design’s adequacy or recommendations for tools and metrics to use would be greatly appreciated!

[#2]

Welcome! I expect you will get a lot of help here.

There are several ways to evaluate a design. The emphasis on some criteria depends on the purpose of the design. You only have two factors, so screening factors does not seem to be a goal. You want to optimize factor settings, so prediction is the primary focus.

In addition to the number of runs, be sure to use a wide range for the continuous factor. This will enhance the chance that you will observe this factor's effect if it is active. It also gives you a more unbiased assessment of the response behavior. That is, please to not set the range solely based on where you expect the optimum setting to be with a narrow range around it.

Do you have an idea of the noise in the response (i.e., a standard deviation)? Do you have an idea of how much changing the factor levels might change the response?

Please see this section from the JMP Design of Experiments Guide about evaluation and then return here with your questions.

[#3]

Hi Mark, thank you for your response. Unfortunately, I have been in this guide before but I have not been able to get an answer to my question. Specifically, I would like to know how can I assess if my design allows to find the variables with the highest influence on the response AND which levels of these variabels allow to maximize it. Additionally, I would like to do it with the minimal number of runs. To do so I have two designs together with its aliasing matrix but I am not able to tell if the design with 12 runs allows to do so:

Besides, I have no idea of the noise in the response nor on how much changing the factor levels might change the response.

Thanks again

[#4]

The initial smaller design will estimate the first order model with some two-factor interactions. The small number of runs does not allow some combinations of factor levels that would reduce or eliminate the correlation between the estimates (Alias Matrix).More runs would allow more combinations and increase the power of the tests for significant effects.

The second, larger design will also estimate curvature in the response to changing X2 and X3, a different model that the one proposed to the first design. The large number of runs also permitted forming combination to eliminate the correlation of the parameter estimates. This maximizes the power of the tests of significance.

Given that you don't know the response standard deviation or effect of these factors a priori. you might run an initial design, learn what you can, and, if necessary, augment the design with new runs using DOE > Augment Design. Your initial experiment can't fail, it can only disappoint you, in which case, you can improve it without starting over or quitting.

[#5]

I'll second everything @Mark_Bailey has contributed. The only other thing I'll add is don't discount measurement system noise as well...that's inherent in the unknown response standard deviation as well. So lacking that knowledge, I'd opt for a small design, bold levels...and learn from there. And as Mark suggests...augment if you haven't solved the practical problem at hand.