Choice of DoE setup

[#1]

I have a question over the setup of a DoE and which might be most appropriate from all the options available.

My experiment relates to maximising the yield of a particular chemical extraction and I have ~5, 2-level parameters both continuous (e.g. time, temp, volume) and categorical (e.g. sample #, shaking on/off).  I would like to have triplicates as the results can be noisy, and I am limited to 24 experiments in one run.  I am considering the merits of centre points also.  My aim is really to find which parameters are critical to the output, and which don't make a difference, rather than try and predict outputs exactly.  I also wonder whether it would be beneficial to split this out into multiple DoEs e.g. eliminate non-critical parameters first then probe more deeply those that are critical.

In JMP you have the options of starting with a custom design, definitive screening, screening, response surface, full factorial, mixture and taguchi.  I can eliminate response surface as I believe that is for only continous parameters.  However for the rest I'm unsure which to start with, and it seems you can arrive at the same point from different routes e.g. screening -> choose from list -> full factorial.  Screening seems like it might be most appropriate but I would be interested to hear thoughts from the community on how one decides which approach is best.

[#2 · Accepted solution]

From your description of the objective, it definitely sounds like you are needing a screening design.  However, I do want to point out that response surface designs are not just for all continuous factors.  You'll eventually want to do one if you ever need to go for good prediction within your design space.

5 factors in 24 runs is certainly do-able for a screening design, though you're not going to be able to replicate every point multiple times.  I'd recommend Custom Design and specifying a model of all main effects and 2-factor interactions.  When I ran 5 factors in 24 runs through there, I got a design with 8 of the points replicated, so you'll have a decent estimate of pure error.

Definitive Screening is probably not right for you unless you want to get something of a response surface at the same time and feel up to the task of attempting a much more complex analysis.  Mixture designs are not relevant here, you don't have enough runs for a full factorial (32 runs for 5 factors), and Taguchi arrays are pretty old school and usually for situations where you need a highly fractionated design (not your case).

In the Screening Design platform, you'll basically have a choice of factorials (full and fractional) and Plackett Burman.  A fractional factorial would only allow powers of 2, so you'd you have to choose either 8, 16, or 32 runs for your design.  At 16 runs, you would have no degrees of freedom for error since you are starting with 16 potential model terms.  You can do that if you're really just looking for the big knobs in your experiment.  Placket Burman is for 12 runs and really only gives you main effects.

You can't go wrong with Custom Design.  It will very often reproduce the classical designs for the same number of runs, but with Custom Design you have complete flexibility over the model you want to estimate and how many runs to include.  For optimality criteria, choose D-optimality for a screening scenario.  That criterion emphasizes power/precise parameter estimates, so you have the highest possible chance of detecting any significant effects for the number of runs you can do.

[#3]

Thank you Cameron, this is very helpful advice :)

[#4]

I forgot to mention that you can include any number of center points in Custom Design.  This will give you the ability to detect curvature in the in a response surface, but not model it.  It didn't sound like that's very important to you at this stage.

[#5]

Having added a few more parameters I find that I get a D efficiency score around 83%.  This appears to drop when I add 1 or more centre points - is this to do with the curvature response, and it's inability to model it accurately?  Would it be best to leave them out - the D efficiency drop is to ~77% but I'm not sure what D efficiency is good/bad, although there is some discussion here: https://community.jmp.com/t5/Discussions/D-Efficiency-and-resolution/td-p/47698

[#6]

D-efficiency is basically a measure of orthogonality. It’s the d-optimality of your design compared to a hypothetical design of the same factors and number of runs that has no correlation between any of its model terms. Most of the time, such a design cannot actually exist.

It’s really just a recognition that you are having to accept more correlation between model terms in order to include all the factors you specified for that number of runs. It’s hard to say what a “bad” d-efficiency is. I think power is a far better metric to judge the utility of a screening design, as well as the color map.

[#7]

Apologies I have one more question about replicates.

Ideally I'd like to run each experiment twice or better three times.  At the end of a custom design, you can select the number of runs from: minimum, default and user specified.  There is a box above that 'Number of Replicate Runs' however entering 1 in that box doesn't double the number of runs.  For an original 8 & 12 for minimum and default it increases to 9 and 16.  Under Classical -> screening design, adding a 1 to the replicates box does indeed double the number of runs.  Why would this differ for the Custom Design?  Also importantly could I set the replicates to 0 for the custom design and just copy paste all the proposed experiments, to allow them all to be replicated?

[#8]

If it is an aqueous/organic chemical extraction be sure to consider brine (NaCl) as a factor. Saturated brine washes are vey effective at helping the partitioning.