How to reduce the number of runs in design?

[#1]

Dear sir/madam,

In JMP17 I have made a design starting from 8 2-level factors, of which 5 are hard-to-change. For the hard-to-change factors only main effects are considered, for the easy-to-change factors 2nd order interactions are also evaluated.

Using Custom Design, 32 runs were generated, consisting of 8 whole plots.

Now, every run represents an adhesive formulation I would like to test on a series of different substrates (hard wood, plywood, …)

Unfortunately, for a limited number of substrates I do not have enough material to perform 32 tests. For those substrates I do not have many of, I would like to test them on a reduced number of runs (or adhesive formulations)

My question is, how can I best select a reduced number of runs? My initial idea was to drop one easy-to-change factor combination per whole plot, reducing the number of runs from 32 to 24. But still, how to choose which easy-to-change factor combinations to remove and which I should better retain?

Thank you in advance for any assistance,

Best regards,

[#2]

Hi @SamVA,

I might have misunderstood something, because trying to reproduce the design you intended with only main effects for 5 hard-to-change factors and main effects + 2-factors interactions for 3 easy-to-change factors leads me to a recommended design with 24 runs and 8 whole plots : 

On a side note, in a split-plot design you can estimate 2-factors interactions between easy-to-change factors, as they are fully randomized compared to hard-to-change factors which have a restriction on their randomization.
Depending on your need and precision estimation of main effects + 2-factors interactions for easy-to-change factors, specifying a design with only main effects when creating your split-plot design with your 8 factors could help allocating the aliases on the interactions involving hard-to-change factors :

You can see in the main effects split-plot design that main effects are completely uncorrelated/non-aliased, and so are the 2-factors interactions for easy-to-change factors (from X6 to X8).

I attached the two designs generated so that you can have a look and compare the two design options,

[#3]

Dear @Victor_G 

Thank you very much for your reply,

It is indeed true that the recommended design for 8 factors (5 hard-to-change, 3 easy-to-change, only ME for HTC and ME+2nd for ETC) gives 24 runs and 8 whole plots.

However, when I increased the number of runs to 32, so one additional easy-to-change factor combination per whole plot, I noticed an improvement in the prediction variance, a reduction in the average correlation and a better D-efficiency. Since doing an extra experiment per whole plot is feasible, and for most substrates there is more than enough material to execute all experiments, I opted to go for the design with 32 runs.

Unfortunately, for 1 substrate I lack sufficient material to do all 32 runs. So, the idea was to select for this 1 substrate only 24 runs out of the 32 in the design (remember, in this experiment every run corresponds to an adhesive formulation). Then the question still is, how to best select which runs to drop, and which to retain for that one substrate.

As an alternative I could go for the recommended design of 24 runs for all substrates, but it seems to be a pity that I lose the improvements associated with the 8 extra runs. Unless of course, and that is for me difficult to assess, that the benefits in terms of effect estimation and significance testing when going to 32 runs would only be marginally better than that of the 24-run design.

Custom design 2 = 32 runs

Custom design 3= 24 runs

[#4 · Accepted solution]

Hi @SamVA,

What is your objective with this design ? If your objective is about exploring and understanding the links between your factors and your response(s), I'm not sure prediction variance is the most interesting indicator for your design (since you also seems interested in D-efficiency). Moreover, the change you see in the Fraction Design Space plot looks very marginal, so I doubt this slight improvement will create a meaningful practical impact in your experiments and system understanding. 

I might have different ideas based on the limited information about your topic :

• If you have a physical experimental constraint limiting your experimental budget (like quantity of substrate available), I would go with the 24-runs design to have a first understanding of the system, before augmenting the design with new runs to increase estimate precision and reduce prediction variance where/when it does matter. It is often more interesting to run DoEs sequentially, instead of trying to solve all the answers with the biggest DoE available.
  Run a first design, visualize and analyze the results, and iterate based on the knowledge gained.
• I'm not sure how practically feasible this idea might be, but adding whole plots (besides runs) can considerally improve your design (if possible/feasible). This situation is easy to understand, the more whole plot you have, the closer your design looks like a fully randomized design, so you improve power for estimating effects, reduce prediction variance, etc...
  Depending on how feasible this option could be, you may try different designs involving a different number of whole plots and different number of runs, to be able to find the best compromise.

Hope these ideas may help you,

[#5]

Dear @Victor_G ,

Every whole plot represents a days work, so it would like to avoid adding more whole plots. Instead, it will follow your suggestion and go for the 24-runs design.

Thanks again for the feedback,

Best regards,

Sam