How select the best design

[#1]

Dear all,

We would like to make experiments with 5 continuous factors. Our system is a chemical reaction. We know there are interactions between factors but we don’t know to which extend. We would like to maximize the amount of product.  

There are a lot of possible DOE’s. I think about

-central composite design,

-OR first part of it (for main effects and central points) and then augment the design,

-OR DSD and then augment the design

-OR custom design with 2d order and quadratic interactions

-OR custom design and select RSM.

How can I select the best design for us? Which argument would help me to choose?

Thanks

[#2]

I bet that all 5 factors are important and you already know that from prior knowledge of the chemical reaction. If so, then I would not start with a screening experiment because the key screening principles are likely not to hold. There is uncertainty about the factor effects, however, so you might use the Estimability attribute of model terms to specify that uncertainty.

It will probably come down to your budget of runs. So I would make a custom design for the 5 factors. (Are any factors hard to change?) I would click the RSM button in the Model section. It will enter terms in the model for all possible second-order interactions and non-linearities. (This action will also change the criterion to I-optimal.) Now inspect each term in the model. Ask yourself if you know that this term is necessary, you know it is not necessary, or you think it might be necessary. Remove any term that you know is not necessary. Leave all terms that you know are necessary alone. Change the estimability of the remaining terms to If Possible. This process will result in two changes. The minimum number of runs will fall - this number is equal to the number of parameters that must be estimated. Also, the design will be I-optimal for all possible models. Now increase the number of runs in the following way. Add 3-4 runs for every If Possible term that might be significant. Say that I have 10 terms for interactions that might be possible, I don't know which ones will be active, but I think that 2 of them will be active. Add 6-8 runs to the minimum.

Does this reasoning and approach make sense?

[#3]

Thanks Mark.

Indeed, I know all these factors are important. However, I don’t know how they interact over their range of values.

Moreover, for one factor, I don’t know the minimum value which is necessary to get a significant amount of product. Therefore, the selected range could be too large: maybe the lowest selected value will give a low response, whatever the values of the other factors.

In the approach you suggest, I understand the aim is to lower the cost of the experiments. However, we don’t want to miss any effect. Therefore, I would choose to keep all the runs. With 3 central points (asked by a partner), considering 2nd order and quadratic terms gives 28 runs; RSM gives 27 runs and a lower Prediction Variance Profile. Therefore, RSM looks to be the best one.

[#4]

You want aggressive ranges for the factors that will provide a large change in the response. That is an advantage of continuous factors that is not possible with categorical factors. The point is that you want high power and small confidence intervals. A test is about finding the winner. An experiment is about modeling. The model will find the winner. So an experiment that produces both good and bad responses is the best kind. Your model will be more realistic.

[#5]

There are a number of considerations when choosing a design.  No one knows which will actually perform better until after you run the experiment.  My advice is to create multiple designs, list pros and cons of each, what could be learned from each (what is separated, what is confounded and what is restricted) and weigh the possible knowledge gained against the resources required.  

One thing I don't see being addressed is noise.  There are a couple of questions:

1. How confident are you in the measurement system?  Repeated runs may help assess this in the context of the experiment.

2. What are your strategies to maximize the likelihood the experiment represents future conditions?  For example, the chemicals used in the chemical reaction are likely from continuous processes that are distributed in batches.  How much variation is there within batch and between batch?  Do ambient conditions have an effect?  Are you concerned with setup of the process?

Consider blocks or split-plots to manage the precision of the experiment while not compromising the inference space.

3. I suggest creating a predicted rank order of model effects up to 2nd order factorial and 2nd order polynomial.  This will help in deciding the resolution you need and whether you consider the departure from a linear model to be significant and therefore need to add levels to estimate this.

4. Lastly, predict every possible outcome and anticipate what you will do with the information gained from each outcome.  For example, if you run the experiment and create a practically significant amount of variation, but none of it is assignable to the factor effects, what will you do?  If factor A is significant and the + level is better, what will you do?  If factor A is significant and - level is better, what will you do? If factor A is insignificant, what will you do?  etc.

[#6]

Hello Statman, I think points 1 and 2 are ok. We made pre-tests regarding the measurement system, and we prepared the tests in a way to lower noise as much as possible. Thanks a lot for your other suggestions.

[#7]

You don't want to lower the noise per se, you want the noise to be representative of the real/actual noise in the process. Of course, as the noise becomes more representative, it can negatively effect the precision of detecting factor effects. That is when you need a noise strategy (e.g., blocks, split-plots) to partition the noise.

“Unfortunately, future experiments (future trials, tomorrow’s production) will be affected by environmental conditions (temperature, materials, people) different from those that affect this experiment…It is only by knowledge of the subject matter, possibly aided by further experiments (italics added) to cover a wider range of conditions, that one may decide, with a risk of being wrong, whether the environmental conditions of the future will be near enough the same as those of today to permit use of results in hand.”
Dr. Deming

"Block what you can, randomize what you cannot"
G.E.P. Box