Introducing definitive screening designs

[#1]

In my two previous posts, I introduced the correlation cell plot for design evaluation and then showed how to use the plot to compare designs. Here, I want to use the same plot to show why definitive screening designs are, well, definitive.

For a complete technical description of definitive screening designs, you can read "A Class of Three-Level Designs for Definitive Screening in the Presence of Second-Order Effects" -- an article I co-wrote with Chris Nachtsheim of the University of Minnesota. Chris and I were delighted to learn recently we had won the American Society for Quality's 2012 Brumbaugh Award for our paper. This award is presented to the author(s) of the paper that has made the largest single contribution to the development of industrial application of quality control. The paper was published in January 2011 in the Journal of Quality Technology, and you can read it via the JMP website.

What is a definitive screening design?

The most notable way that definitive screening designs are different from standard designs is that all the factors are numeric and are tested at three levels. A second distinctive feature of a definitive screening design is that it is a self-foldover. That is, the runs of the design come in pairs that “mirror” each other. Suppose we encode the low setting of a factor as “–“, the high setting as “+” and the middle setting as “0”. Then, if one run of a foldover pair has factor settings encoded “+ 0 – + – +”, the other run has factor settings encoded “– 0 + – + –”. Each pair of runs has one factor at its middle value and all the others at their high or low values. One run is at the center of the design region with all the factors at their middle setting. Table 1 shows a definitive screening design for eight factors. Notice that it has one more than twice as many runs as there are factors, that is, 17 runs.

Table 1 Definitive screening design for eight three-level factors.

So what makes this design so special?

To see why the design in Table 1 is fantastic, let us use the correlation cell plot in Figure 1. Our potential model terms are all the main effects, two-factor interactions and quadratic effects. Note that only the cells on the diagonal of the plot are pure red. That means that none of the model terms are confounded with each other.

Figure 1 Correlation plot for definitive screening design.

The last eight columns of the cell plot show the quadratic effect terms. These effects are only mildly correlated with each other (|r| = 0.19). Each quadratic effect is uncorrelated with a two-factor interaction involving its factor. That is, the quadratic effect of factor A is uncorrelated with the AB interaction. Other two-factor interactions have an absolute correlation of 0.37. It turns out that all eight quadratic effects are estimable with the definitive screening design. The main effects of the design are all orthogonal to each other and to all the second order terms (two-factor interactions and quadratic effects).

The two-factor interactions have pairwise correlations that can take one of three values. The pink cells represent absolute correlations of two-thirds. The light blue cells represent correlations of only one-sixth. The pure blue cells show uncorrelated interaction pairs.

Let us compare this design and plot to the standard screening design for eight factors. That design is the minimum aberration fractional factorial design. This design is in Table 2, which has one added center run to make both designs have 17 runs including one center run.

Table 2 Standard screening design with one center run.

Figure 2 shows the cell plot for the fractional factorial design. The most notable feature of this cell plot is that all the cells are either pure blue or red. That is, every pair of columns is either completely uncorrelated or completely confounded.

Figure 2 Correlation plot for the standard screening design.

Note the block of red cells in the lower right. These red cells indicate that all the quadratic effects are confounded with each other. With one added center run, the standard screening design has some ability to detect very strong nonlinearity in the factor/response relationship. However, there is no way to determine which factor is causing the nonlinearity. By contrast, the definitive screening design can separately estimate the nonlinear effect of each factor.

Each two-factor interaction in the fractional factorial design is confounded with three other two-factor interactions. This means that if any two-factor interaction is active, the analysis can only indicate that there are four possible interactions that could explain the observed effect. Narrowing down this field to one interaction requires further experimentation. By contrast, the definitive screening design can reliably resolve any two-factor interaction that is large compared to its standard error.

Why are definitive screening designs definitive?

The purpose of screening is to separate the vital few factors that have a substantial effect on the response from the trivial many that have negligible effects. If a factor’s effect is strongly curved, a traditional screening design may miss this effect and screen out that factor. If there is a two-factor interaction, standard screening designs having a similar number of runs to the definitive screening design with the same number of factors will require follow-up experimentation to resolve the ambiguity. The definitive screening design can reliably accomplish the task of screening even if there are a couple of second order effects.

Try the definitive screening designs add-in available from the JMP File Exchange (download requires a free SAS profile). The add-in works with JMP 9 and JMP 10.