Regarding Repeat Experiments in DoE

[#1]

Hello All,

I have what I am hoping will be a simple problem for the group.

I recently set up a DoE with three factors, each having discrete numerical values at 3 levels.  I specified cross interactions with 2nd order effects with the 18 runs to be executed in 3 blocks.  I did NOT specify any repeats to be designed, however, I notice in the design matrix that 4 of the runs are duplicated.  In other words: the 18 run design comprises 14 unique combinations and 4 which are duplicates.

First: is this atypical?  I had thought that the 18 runs should all be unique.  Is there some kind of powering that JMP is trying to achieve with this?

-or-

Second: could there be something that I'm missing in my initial setup and we're actually NOT really probing the design space correctly?

Unfortunately the first two blocks have been initiated and include ONE of the repeats (i.e., I have 11 unique runs with 1 repeat, so far). 

If I assume that the design is appropriate...it follows that we can continue according to the original plan and evaluation (correct?).

However, if the feedback is such that this ISN'T the best approach: 1) what would we be missing if we continued with the original design?  2) Is it possible to reconfigure the last random block to substitute alternative conditions for the repeats?

Thanks in advance for your input on this!

Chris 

[#2]

Custom Design provides many features to tailor the result to your situation. They can have profound effects on the resulting design. You are thinking about combinations (3x3x3x2), but JMP is not doing that. Why not? Because that combinatorial result ('full factorial design') might not be optimal. You added random blocks of size 6, so you cannot have balance in the design. You used Discrete Numeric factors to force three levels in the plan for each factor. That choice constrains the design. It must add quadratic terms to the model to keep three levels in the design but changes their estimability to 'if possible.' That forces the use of a Bayesian D-optimal method. So you are asking JMP to optimize a design to estimate a set of models. As a result of all these specifications, you get some replication because it is more optimal than the design without replication.

Do you intend to estimate terms for curvature? Do you instead want to test for curvature and then augment the design if it is significant?

I created two designs based on your description, as I understand it. The second one uses continuous factors, but otherwise, it is the same. I attached both of the data tables for your exploration.

[#3]

 Thanks, Mark, for your input.  

Perhaps I set up the design incorrectly.  I'm not married to the discrete levels for my factors...we are trying to find optimal conditions and can accommodate continuous factors.  So - yes - we would like to estimate curvature and then confirm the results with additional runs as necessary.

With that in mind: would you recommend altering the last block of 6 runs to replace the replicate runs with others (I cannot change the first 2 blocks and they include only 1 replicate)?  Am I able to alter the design in JMP at this point to accommodate any of the above?

Thanks for the two designs - I'll have a look at them.

Regards.

Chris

[#4]

Hi @CBroomell1,

If you can provide context about the factors, that would help to give you an advice about which type would suit the factors better in your design. If you have numerical factors but you can't fix any value between some intervals (for example, industrial furnace temperature may be considered as a discrete numeric factor, where you can switch the level between 180, 200 and 220°C but not between these values as the "step size" is 20 for example). If you have no constraints and can choose any numerical value between your levels, then you may have a continuous numerical factor.

You have different ways, from the curvature testing to quadratic terms full estimation, to take into account the search and estimation of quadratic terms.

1. The basic option is to add centerpoint(s) to assess if there is any curvature in your response(s). This economic approach enables to do a lack-of-fit test for curvature, and if a significant lack-of-fit is detected, you can fit one quadratic effect (and one only) in the model thanks to the centerpoint(s). You can assess which quadratic term might be the most important to add in the model with analysis like Stepwise Regression. 
2. In the case of discrete numeric factor, JMP by default enter the quadratic terms in the model and set their estimability to "If possible", meaning that if you have enough degree of freedom in your model, then quadratic terms will be estimated. You can also do this with numerical continuous factor, entering them in the model and switching their estimability from "Necessary" to "If possible" by clicking on the text. This method provides a flexible way to estimate quadratic terms for non-saturated designs (with number of runs bigger than the minimum required).
3. You can also directly add in the model quadratic terms and set their estimability to "Necessary". This way, JMP will set up experiments in your design to be able to estimate these effects. If they are not significant, you may drop them in your modeling (and if they also don't make sense from a domain expertise point of view).

Concerning your question, I wouldn't modify "on the run" the DoE, but would look at possible augmentations based on the findings from this first DoE.

Hope this answer will help you,

[#5]

Thanks Victor!

I won't change the DoE on the fly.

Regarding the parameters: they are actually continuous but with a center point.  I was unclear on how to set up the initial matrix and set three discreet values for each level.  in reality, though, we do have flexibility to wander between certain limits (factors are related to concentration of reagents and time).

So - once we are complete with our final block (using the matrix set up with discrete values) - will we be able to get any curvature in the model or are we basically stuck with the original values until we explore further?

Best,

Chris

[#6]

Hi @CBroomell1,

If you didn't change the estimability of quadratic effects in your original design (set by default to "If possible" by JMP), changing the factor type from discrete numeric to continuous numeric will get you in the second approach I described : if you have enough degree of freedom, since you had a middle level in each of your numeric factor (middle level was "forced" in the design due to the initial factor type discrete numeric), JMP may be able to estimate quadratic terms in the model. 

Since you have 14 unique runs (and 4 replicate runs), you should be able to estimate 14 terms (intercept + 13 terms), so quadratic terms could be estimated in your design.

If you have any problem with the analysis/modeling part, don't hesitate to ask the JMP Community again.

Hope this answers your question,

[#7]

First of all, you can't break it. You will use regression analysis, which will still work. The design is an attempt to collect the optimum data for the model.

I would try to recover in this way. Delete the runs that have not been executed yet. Then select DOE > Augment Design. This platform will create the best set of runs to finish, given the runs you already have. Try it.

[#8]

Hi @Mark_Bailey,

Since the question was about estimating quadratic terms and switching discrete numeric factors to continuous numeric factor, I don't see what would be the advantage of modifying the design "on the fly" since these changes can be achieved without modifying any runs (and without the risk of dropping the random block in the augmentation) ? Estimation of quadratic terms was already possible from the first design, so I don't see the point of changing the runs from the last random block ?

I know JMP is able to comply with changes during experiments with the "Augment design" platform, no "technical" doubt about this (and it can be indeed a good example to "play" with the "Augment design" platform and get familiar with it), but here I don't see any concrete and significant advantage of not running the already planned last random block of experiments and changing these runs by newly computed ones : the comparison of the two approaches shows two very similar designs, so it seems better (or at least easier) to stick with the "simple" and pragmatic option to follow the original design ? Can you please explain if I'm missing the point ?

Thanks in advance,

[#9]

I apologize for the confusion. The background was not entirely transparent initially, and replies got out of synch as more information emerged. I was not suggesting a change mid-stream. I was trying to explain how Custom Design could arrive at an unexpected solution. I was trying to present an alternative approach from the beginning. That is all.

[#10]

No problem @Mark_Bailey, I got confused reading your response as I fear I might have missed something.
I got the same problem of sync with the first reply on this post, as I thought giving the first answer when writing, but when my reply was posted, you already had proposed an answer.