Introducing Definitive Screening Designs

[#1]

Recently the American Society for Quality (ASQ) named the winners of the 2012 Brumbaugh Award. This award is presented for the paper that has made the largest single contribution to the development of industrial application of quality control. This year’s winning paper was "A Class of Three-Level Designs for Definitive Screening in the Presence of Second-Order Effects." It was published in January 2011 in the Journal of Quality Technology.

I use the correlation cell plot here to show why definitive screening designs are definitive.

What is a definitive screening design?

The most notable way that definitive screening designs differ from standard designs is that all the factors are numeric and tested at three levels. A second distinctive feature of a definitive screening design is that it is a self-foldover. That is, the runs of the design come in pairs that “mirror” each other. Suppose we encode the low setting of a factor as –, the high setting as +, and the middle setting as 0. Then, if one run of a foldover pair has factor settings encoded + 0 – + – +, the other run has factor settings encoded – 0 + – + –. Each pair of runs has one factor at its middle value and all the other factors at their high or low values. One run is at the center of the design region with all the factors at their middle setting. Table 1 shows a definitive screening design for eight factors. Notice that it has one more than twice as many runs as there are factors, that is, 17 runs.

So what makes this design so special?

To see why the design in Table 1 is fantastic, let us use the correlation cell plot in Figure 1. Our potential model terms are all the main effects, two-factor interactions and quadratic effects. Note that only the cells on the diagonal of the plot are pure red. That means that none of the model terms are confounded with each other.

The last eight columns of the cell plot show the quadratic effect terms. These effects are only mildly correlated with each other (|r| = 0.19). Each quadratic effect is uncorrelated with a two-factor interaction involving its factor. That is, the quadratic effect of factor A is uncorrelated with the AB interaction. Other two-factor interactions have an absolute correlation of 0.37. It turns out that all eight quadratic effects are estimable with the definitive screening design. The main effects of the design are all orthogonal to each other and to all the 2^nd order terms (two-factor interactions and quadratic effects).

The two-factor interactions have pairwise correlations that can take one of three values. The pink cells represent absolute correlations of two-thirds. The light blue cells represent correlations of only one-sixth. The pure blue cells show uncorrelated interaction pairs.

Figure 1 Correlation plot for definitive screening design.

Let us compare this design and plot to the standard screening design for eight factors. That design is the minimum aberration fractional factorial design. This design is in Table 2, which has one added center run to make both designs have 17 runs, including one center run.

Figure 2 shows the cell plot for the fractional factorial design.

The most notable feature of this cell plot is that all the cells are either pure blue or red. Every pair of columns is either completely uncorrelated or completely confounded.

Note the block of red cells in the lower right. These red cells indicate that all the quadratic effects are confounded. With one added center run, the standard screening design has some ability to detect very strong nonlinearity in the factor/response relationship. However, there is no way to determine which factor is causing the nonlinearity. By contrast, the definitive screening design can separately estimate the nonlinear effect of each factor.

Each two-factor interaction in the fractional factorial design is confounded with three other two-factor interactions. This means that if any two-factor interaction is active, the analysis can only indicate that there are four possible interactions that could explain the observed effect. Narrowing down this field to one interaction requires further experimentation. By contrast, the definitive screening design can reliably resolve any two-factor interaction that is large compared to its standard error.

Why are definitive screening designs definitive?

The purpose of screening is to separate the vital few factors that have a substantial effect on the response from the trivial many that have negligible effects. If a factor’s effect is strongly curved, a traditional screening design may miss this effect and screen out that factor. If there is a two-factor interaction, standard screening designs having a similar number of runs to the definitive screening design with the same number of factors will require follow-up experimentation to resolve the ambiguity. The definitive screening design can reliably accomplish the task of screening even if there are a couple of 2^nd order effects.

Run	A	B	C	D	E	F	G	H
1	0	1	1	1	1	1	1	1
2	0	-1	-1	-1	-1	-1	-1	-1
3	1	0	-1	-1	1	-1	1	1
4	-1	0	1	1	-1	1	-1	-1
5	1	1	0	-1	-1	1	-1	1
6	-1	-1	0	1	1	-1	1	-1
7	1	1	1	0	-1	-1	1	-1
8	-1	-1	-1	0	1	1	-1	1
9	1	-1	1	1	0	-1	-1	1
10	-1	1	-1	-1	0	1	1	-1
11	1	1	-1	1	1	0	-1	-1
12	-1	-1	1	-1	-1	0	1	1
13	1	-1	1	-1	1	1	0	-1
14	-1	1	-1	1	-1	-1	0	1
15	1	-1	-1	1	-1	1	1	0
16	-1	1	1	-1	1	-1	-1	0
17	0	0	0	0	0	0	0	0

Table 1 Definitive screening design for eight three-level factors.

Run	A	B	C	D	E	F	G	H
1	-1	-1	-1	-1	-1	-1	-1	-1
2	-1	-1	-1	1	1	1	1	-1
3	-1	-1	1	-1	1	1	-1	1
4	-1	-1	1	1	-1	-1	1	1
5	-1	1	-1	-1	1	-1	1	1
6	-1	1	-1	1	-1	1	-1	1
7	-1	1	1	-1	-1	1	1	-1
8	-1	1	1	1	1	-1	-1	-1
9	1	-1	-1	-1	-1	1	1	1
10	1	-1	-1	1	1	-1	-1	1
11	1	-1	1	-1	1	-1	1	-1
12	1	-1	1	1	-1	1	-1	-1
13	1	1	-1	-1	1	1	-1	-1
14	1	1	-1	1	-1	-1	1	-1
15	1	1	1	-1	-1	-1	-1	1
16	1	1	1	1	1	1	1	1
17	0	0	0	0	0	0	0	0

Table 2 Standard screening design with one center run.

[#2]

Figure 2 shows the cell plot for the fractional factorial design.

The most notable feature of this cell plot is that all the cells are either pure blue or red. Every pair of columns is either completely uncorrelated or completely confounded.

Note the block of red cells in the lower right. These red cells indicate that all the quadratic effects are confounded. With one added center run, the standard screening design has some ability to detect very strong nonlinearity in the factor/response relationship. However, there is no way to determine which factor is causing the nonlinearity. By contrast, the definitive screening design can separately estimate the nonlinear effect of each factor.

Each two-factor interaction in the fractional factorial design is confounded with three other two-factor interactions. This means that if any two-factor interaction is active, the analysis can only indicate that there are four possible interactions that could explain the observed effect. Narrowing down this field to one interaction requires further experimentation. By contrast, the definitive screening design can reliably resolve any two-factor interaction that is large compared to its standard error.

Why are definitive screening designs definitive?

The purpose of screening is to separate the vital few factors that have a substantial effect on the response from the trivial many that have negligible effects. If a factor’s effect is strongly curved, a traditional screening design may miss this effect and screen out that factor. If there is a two-factor interaction, standard screening designs having a similar number of runs to the definitive screening design with the same number of factors will require follow-up experimentation to resolve the ambiguity. The definitive screening design can reliably accomplish the task of screening even if there are a couple of 2^nd order effects.

Run	A	B	C	D	E	F	G	H
1	0	1	1	1	1	1	1	1
2	0	-1	-1	-1	-1	-1	-1	-1
3	1	0	-1	-1	1	-1	1	1
4	-1	0	1	1	-1	1	-1	-1
5	1	1	0	-1	-1	1	-1	1
6	-1	-1	0	1	1	-1	1	-1
7	1	1	1	0	-1	-1	1	-1
8	-1	-1	-1	0	1	1	-1	1
9	1	-1	1	1	0	-1	-1	1
10	-1	1	-1	-1	0	1	1	-1
11	1	1	-1	1	1	0	-1	-1
12	-1	-1	1	-1	-1	0	1	1
13	1	-1	1	-1	1	1	0	-1
14	-1	1	-1	1	-1	-1	0	1
15	1	-1	-1	1	-1	1	1	0
16	-1	1	1	-1	1	-1	-1	0
17	0	0	0	0	0	0	0	0

Table 1 Definitive screening design for eight three-level factors.

Run	A	B	C	D	E	F	G	H
1	-1	-1	-1	-1	-1	-1	-1	-1
2	-1	-1	-1	1	1	1	1	-1
3	-1	-1	1	-1	1	1	-1	1
4	-1	-1	1	1	-1	-1	1	1
5	-1	1	-1	-1	1	-1	1	1
6	-1	1	-1	1	-1	1	-1	1
7	-1	1	1	-1	-1	1	1	-1
8	-1	1	1	1	1	-1	-1	-1
9	1	-1	-1	-1	-1	1	1	1
10	1	-1	-1	1	1	-1	-1	1
11	1	-1	1	-1	1	-1	1	-1
12	1	-1	1	1	-1	1	-1	-1
13	1	1	-1	-1	1	1	-1	-1
14	1	1	-1	1	-1	-1	1	-1
15	1	1	1	-1	-1	-1	-1	1
16	1	1	1	1	1	1	1	1
17	0	0	0	0	0	0	0	0

Table 2 Standard screening design with one center run.