21st Century Screening Designs (2020-US-45MP-538)

Americas 2020

21st Century Screening Designs (2020-US-45MP-538)

Bradley Jones, JMP Distinguished Research Fellow, SAS

JMP has been at the forefront of innovation in screening experiment design and analysis. Developments in the last decade include Definitive Screening Designs, A-optimal designs for minimizing the average variance of the coefficient estimates, and Group-orthogonal Supersaturated Designs. This tutorial will give examples of each of these approaches to screening many factors and provide rules of thumb for choosing which to apply for any specific problem type.

Auto-generated transcript...

Speaker	Transcript
Bradley Jones	Thanks for joining me.
	I'm going to give a talk on 21st century screening designs.
	The topic of screening designs has changed over the last 20 years.
	I'd like to talk about the three kinds of screening designs that I would recommend, all of which are available in JMP.
	A-optimal screening designs, definitive screening designs (DSDs), and group orthogonal super saturated screening designs (GO SSDs).
	Now that might be a little surprising to you because the default in the custom designer has been D-optimal for around 20 years.
	However, over the last few years I've come to the conclusion that A-optimal designs are better than D-optimal designs. I'm going to be illustrating that point in the first section of this talk.
	Those are the three designs I'm comparing. I'm going to start with A-optimal designs. My slide has a graph showing exactly why I think that A-optimal designs might, in many cases, be better than D-optimal screening designs.
	The example is a four-factor 12-run design for fitting all the main effects and all the two-factor interactions of four continuous factors. You can see that for the A-optimal design here...
	I'm showing there are only three cells where there are non-zero correlations in the correlation cell plot. The rest of the pairwise correlations are all zero for the A-optimal design.
	For the D-optimal design, there are a lot of correlations all over the place and that makes model selection harder.
	This concludes my first example. What is A-optimal, you might ask? What does the �A� in A-optimality stand for?
	The �A� stands for the average variance of the parameter estimates for the model.
	The parameter estimates are the estimates of the four main effects and the six two-factor interactions. If you minimize the average of them, you tend to do a reasonable job at lowering every one of them, at least doing better than some other approaches.
	The way to remember what A-optimal means is that the �A� stands for average, the average variance of the parameter estimates.
	Bradley Jones
Bradley Jones	That makes the A-optimality criteria easy to understand. Everybody understands what an average is. We have all these estimates. We want the variances of those estimates to be small in general and that's what the A-optimality criterion does in a direct way.
	The other nice thing about A-optimal designs is that A-optimal designs can allow for putting different emphasis on groups of parameters through weighting.
	Now you could weight D-optimal designs, but the weighting on a D-optimal design doesn't change the design, so it's kind of useless.
	Whereas when you weight A-optimal designs, you get differential weighting on the parameters. You might say, well, main effects are more important to me than two-factor interactions, so I want to weight them higher. We have features in JMP for doing just that.
	I have two different A-optimal design demonstrations. One is the one I just showed you
	that is, the output for four-factors, 12-runs D-optimal versus A-optimal for all the two-factor interactions and main effects. The second example is a five-factor experiment with 20 runs and a model having main effects and all the two-factor interactions.
	With D-optimal versus a weighted A-optimal where I put more weight in the A-optimal design on the main effects. Let me switch over to JMP.
	Here's the D-optimal design for four factors and 12 runs and here's the A-optimal design for the same problem. And now I want to, I want to compare these two designs.
	If we look at the relative estimation efficiency, the A-optimal design has higher...estimation efficiency for every parameter except this...
	except for the parameter that is the interaction between X1 and X3.
	If we look at the average correlations for the A-optimal design, the average correlation is only 0.02 whereas...
	for the D-optimal design, the average absolute correlation is 0.116. So there is almost six times as much correlation in the D optimal design as in the A-optimal design.
	The A-optimal design isn't quite as D efficient
	As the D-optimal design. Of course, the D-optimal design has been chosen to be the most D efficient that you can be. The fact that A-optimal design is still
	97+% D efficient is really good. But look, the A-optimal design is 87.5% more G efficient than the D optimal design. So the A-optimal design is reducing the worst possible variance of prediction.
	Of course, the A efficiency of the A-optimal design is 14.5% more efficient than A efficiency of the D optimal design.
	And the I efficiency of the A-optimal design is 16% more efficient than the D-optimal design. All of these efficiency measurements and all of these correlations
	make it pretty clear to me that the A-optimal design in this case is far better than the D optimal design. And that's one of the reasons why A-optimality is now the default for screening including two-factor interactions in JMP 16.
	Let's look at the second example. Here I have a five-factor experiment. Here's the five-factor 20-run D-optimal design and then a five-factor
	20-run weighted A-optimal design. Now let me show you...I'm going to show you the...the JMP script that creates this design. And when I point out is these parameter weights. This weight factor is saying, I want to weight the intercept and the five
	main effects
	100 times higher than the
	10 two-factor interactions. Let's see what that does to the design. Now we'll compare designs.
	And see, here is the efficiency of the D-optimal design with respect to the weighted A and the weighted A is estimating the five main effects all better than the D-optimal design, because the D-optimal design is
	Bradley Jones
Bradley Jones	this 97% through 99%
	is the relative efficiency of the D optimal to the weighted A-optimal for estimating the main effects. The effect of weighting the main effects is that you get a little bit less good
	variance for the two-factor interactions. Here you're making your estimation of the main effects better at the expense of making your estimation of the two-factor interactions a little worse. But that is what you wanted by weighting the main effects so heavily.
	Again, here the average absolute pairwise correlations among the
	for the weighted A is about 50% better than the those for for the D optimal design. And again, if we look at A-optimal correlation cell plots versus the D-optimal correlation cell plots, you can see that there are a lot more
	there are a lot more white cells in the A-optimal plot, which means these
	these pairwise correlations are zero. There are a lot of zeros here. And that means that it will be a lot easier to separate the, the true active effects from the inactive effects.
	The D-optimal design is only a half a percent better than the A-optimal design, even on its own criteria, but the A efficiency
	is right at nominal and the A-optimal design is more G efficient and also more I efficient than the D-optimal design. Now here, it might be a little bit harder to say which one you would choose. For me,
	I said I wanted to be able to estimate main effects better and I can.
	Those are my two demonstrations for the A-optimal design. Let me go back to my slides.
	Okay, so what I want to do for each of these three kinds of designs is give you an idea about when you would use one in preference to the others.
	For the A-optimal screening design, A-optimal designs and the other designs supported in the custom designer are more flexible than other designs. They allow you to solve more different kinds of problems. In particular,
	you would use an A-optimal screening design if you have a lot of categorical factors, especially if there are more than two levels.
	If there are certain factor level combinations that are not feasible, like for instance if you couldn't do the high setting of all the factors,
	then you would you want to say, I want an A-optimal design because the other...the other two screening designs wouldn't be able to do that.
	And when you have a particular model in mind that you want to fit that's not just the main effects or the main effects plus two-factor interactions,
	you would use the A-optimal screening design, rather than the definitive screening designs or a group orthogonal supersaturated design.
	Moreover, you would use an A-optimal design if you want to put more weight on some group of effects than other groups of effects, and I showed you an example of that where I wanted to put more weight on the main effects
	rather than the two-factor interactions.
	Let's move on to definitive screening designs now. Definitive screening designs first appeared in a published
	journal article in 2011 and they appeared in JMP
	roughly the same time.
	They are a very interesting design. You can see, looking at the correlation cell plot here that
	in a definitive screening design, the main effects are all orthogonal to each other. That is, there is no pairwise correlation between any pair of main effects, but also the main effects are
	uncorrelated with all the two-factor interactions and all the quadratic effects. So that makes it very easy to see which main effects are important.
	Because definitive screen designs have far fewer runs than a response surface design, there are correlations between two-factor interactions
	and between two-factor interactions and quadratic effects and among the quadratic effects. But notice each quadratic effect is uncorrelated with all the two-factor interactions that have...
	this is, this is the quadratic effect of X1 squared.
	Bradley Jones
Bradley Jones	that effect is completely orthogonal to all of these effects that have X1 in the two-factor interaction. And that's the same for all of the factors. The squared term for X2 is orthogonal to all of the effects that have X2 in them and so on.
	That's an interesting property that turns out to be quite useful. What does the DSD look like? Well
	they have three main properties that I want to tell you about. The first property is if we look at Run 1 and Run 2.
	In Run 1, whenever there's a +1 in Run 2 there's a -1 and vice versa. Whenever there's a -1 in Run 1, there's a +1 in Run 2 so that
	Run 1 and Run 2 are kind of mirror images of each other.
	The same thing is true of Run 3 and Run 4, Run 5 and Run 6, Run 7 and Run 8, Run 9 and 10, and 11 and 12. So this design is what's called a fold-over design, and that folding over
	is what makes the main effects not be correlated to the two-factor interaction. Of course, I have this design
	sorted in such a way that you can see
	this structure, but when you run this design, you should actually randomize the order of the runs.
	The second thing I want to point out is that for each factor, there are two center runs
	in some pair of runs. For example, in the first pair of runs, A is at its center level; in the second pair of runs, B is at its center level.
	And in the third set of runs, C is at its center level; in the fourth set of runs, D is at its center level and so on.
	And then the last thing to notice is that there's one overall center run. We have six factors, Factors A through F, that's six.
	And the number of runs is 2x6+1. And the model that...one model that we can fit with this design is the model that contains the intercept, all six main effects, and all six
	quadratic effects. So that's 13 different effects in a 13-run design. It's, it's amazingly efficient in terms of the allocation of runs to effects.
	Now, what are the positive consequences of having a definitive screening design? Well first, an active two-factor interaction doesn't affect the estimate of any main effect because they are uncorrelated.
	Which makes it the case that any single active two-factor interaction can be identified uniquely.
	The same thing is true of any single quadratic effect as long as that quadratic effect is large, with respect to the noise.
	And then one really interesting consequence is that if if...it turns out, if only three factors are active, let's say factors A, C and E,
	then I can fit a full quadratic model and those three factors. A full quadratic model is the kind of model that people fit when they're doing response surface methods or RSM methods.
	And it doesn't matter which three factors are the active ones. The DSD will always be able to fit a full quadratic model, no matter which three factors turn out to be active.
	That's a very powerful thing. It�s result is that if you're lucky enough that only three out of the six factors are important, you can skip the RSM step in some cases and do RSM and screening in one fell swoop.
	I've told you all the good things about DSDs. There is one bad thing, or maybe less good thing, and that is because of those...
	because of these zeros in each column, the main effects are not estimated as precisely in a definitive screen design as they are in a design that would be an orthogonal design with one center run.
	As a result, confidence intervals on the main effects for DSD are going to be 10% or around 10% longer than confidence intervals if you had run, say a Plackett-Berman design of the same number of runs.
	That's, that's a very small price to pay, in my view, for all the benefits that you get in, particularly the benefit of being able to fit quadratic effects, which you can't do
	With the Plackett-Berman design having a single center run. You can identify that you need to be able to fit quadratic effects because you have the center run but you don't know which of the factors has the high curvature.
	Now, when would you use a DSD? Well, you use them when most of the factors are continuous, when you...
	and if they're continuous, you might have the factor levels set far enough apart that you're concerned about possible nonlinearity or curvature in the effect of factor on a response.
	And then you're also concerned about the possibility of two-factor interactions, although the DSDs cannot promise you that you can fit all the two-factor interactions,
	because there just aren't enough runs. If there are a couple or three two-factor interactions, you're likely to be able to identify them with a DSD.
	Okay, let's go to back to JMP and
	back to the journal here.
	Bradley Jones
Bradley Jones	This is a DSD. It's the DSD that has six factors and, instead of 13 runs, I created the eight-factor design and just dropped the last two factors.
	Now when I fit the full factorial model,
	I created a full factorial design for this problem and fit it.
	Bradley Jones
Bradley Jones	Let me show you the parameter estimates.
	The parameter estimates that are
	significant in the full factorial design are A, B, and C, AB, AC, B*C and A squared. Now,
	let me show you the analysis that you would get by doing Fit Definitive Screening. I have time, as my
	as my response, and A through F as my factors.
	And let's look at the model that comes out.
	The model that that this is finding has A, B C and E; E is a spurious effect. So that's a Type I error, but it identified all three two-factor interactions and the quadratic effect of A. The full factorial design that I showed you here had
	three to the sixth runs, which is 729 runs
	This design here only has 17 runs. The fact that I was able to identify all the correct terms with far, far less runs is
	is an eye opener for why definitive screening designs are really great.
	Let's go back to the slides.
	DSDs are great when almost all of the factors are continuous. You can accommodate a couple or three two-level categorical effects. And you can also block definitive screen designs much more flexibly than you can block
	fractional factorial designs. For a six-factor definitive screening designs, you can have anywhere between two and six blocks.
	And the blocks are orthogonal to the main effects. That's, that's another amazing thing about these designs.
	Let's move on to the newest of the screening designs. These have just been discovered in the last couple of years and the publication in Technometrics just came out in the last week or so.
	It's been online for a year, but the actual printed article came to my house in my mail just a week or so ago.
	This is a correlation cell plot of a group orthogonal supersaturated design and you might notice all this gray area.
	In most of the time, if you look at a supersaturated design, the correlation cell plot has correlations everywhere.
	Here we only see correlations in groups of factors. This group of factors is correlated. This other group of factors is correlated.
	This group of factors is correlated and this other group of factors is correlated, but there are no correlations between any pair of groups of factors.
	The only correlations that you see are within a group not between groups, and that helps you with analyzing the data. Here's a pic of the first page the published article,
	which I just said when into print just last week or so. My coauthors are Chris Nachtsheim, this guy here,
	from the University of Minnesota; my colleague from JMP, Ryan Lekivetz; Dibyen Majumdar who's an Associate Dean at the University of Illinois in Chicago in the statistics department; and Jon Stallrich, who is a professor at NC State.
	Let me talk about why you might even be interested in group orthogonal supersaturated designs or supersaturated designs at all.
	And then I'll show how we make a group orthogonal supersaturated design. I will show how to analyze them, except that you don't have to learn how to analyze them because there's an automatic analyze tool in JMP that's right next to the designer.
	And then I'll show you how the two-stage analysis of these �Go SSDs�, as we call them,
	How they compare to more generic analytical approaches. Then I'll make some conclusions.
	What's a super saturated design? A supersaturated design has more factors than runs. For example, you might have something like 20 factors and you only have 12 runs to investigate them in. Then the question you might ask yourself is, "Is this this a good idea?"
	Well, a former colleague of mine, who has since retired
	about 15 years ago or so, told me supersaturated designs are evil.
	Bradley Jones
Bradley Jones	I do understand why he felt that way. The problem with a supersaturated design is that you can't do multiple regression, because
	you have more factors and runs so the matrix that you want to be able to invert is not invertible.
	And then also the factor aliasing is typically complex, although in these group orthogonal supersaturated designs, it's a lot less complex.
	And there's this general feeling that you can't get something for nothing. It feels like you're not putting enough resources into the design to get anything good out of it.
	Bradley Jones
Bradley Jones	Supersaturated designs were first discussed in the literature
	by a mathematician by
	the name of Sattherwaite.
	Bradley Jones
Bradley Jones	His paper was roundly excoriated by a lot of the high-end statisticians of the day.
	Even, you know, laughed at him to a large degree and then three years later Booth and Cox
	Discuss the possibility of systematically generating a supersaturated design and they had a selection criterion which said, look at all of the squared pairwise
	correlations
	so they're all positive numbers and look at the average of that and make that as small as possible.
	even though the design cannot be orthogonal because in order to have an orthogonal design, you have to have more runs than factors. The criterion of Booth and Cox is trying to find the closest to an orthogonal design as it can, given that there are fewer runs than factors.
	We think that John Tukey was the first to use the term "supersaturated" in his discussion of Sattherwaite in 1959.
	Here's what Tukey said, "Of course constant balance can only take us up to saturated...
	saturation (one of George Box's well-chosen terms) up to the situation where each degree of freedom is taken up with the main effect (or something else we are prepared to estimate)."
	As a result, a saturated design has no degrees of freedom leftover to estimate the variance. But Tukey then says, "I think it's perfectly natural and wise to do some supersaturated experiments."
	But in general, the statistics community didn't take that to heart and nothing happened for 30 years after that.
	30 years later, Jeff Wu, who's now a professor at Georgia Tech, wrote a paper in Biometrika, talking about
	one way of making supersaturated designs. And the same year in Technometrics, Dennis Lin, who's now the chair of the Statistics Department at Purdue,
	wrote a paper about another way to create supersaturated designs and both of these papers were very interesting and they brought supersaturated design back into people's consciousness.
	When would you use supersaturated design? Well, one time that you would use it is when runs are super expensive.
	If a run costs a million dollars to do, you don't want to do very many runs. You want to do as few as possible. And if you can do fewer runs than you have factors, all the better.
	I don't know about you, but I've done a lot of brainstorming exercises with stakeholders of processes, and it's very easy when everybody writes a sticky note with a factor they think might be
	active, and you get three dozen sticky notes on the wall, and they're all different.
	Bradley Jones
Bradley Jones	So, what are you supposed to do then? Well, what often happens there is, people are used to doing screening experiments with 6, 7, 8 maybe even 10 factors.
	But people get really nervous doing a screening experiment that has three dozen factors. And so, what happens is, after this brainstorming session happens, the engineers decide, well, we can get rid of
	maybe 20 of these factors
	because we know better than the people who pick those 20 factors.
	Bradley Jones
Bradley Jones	What I'm afraid of is that when you do that you might be throwing the baby out with the bathwater, so to speak.
	The, the most important factor might be one of those 20 that you just decided to ignore.
	And the factors that you end up looking at may look like there's a huge amount of noise because they're not taking account of this other more important factor that was left out.
	I think that eliminating factors without any data is unprincipled. It's, definitely not a statistical approach.
	Now, how do we construct these Go SSDs?
	Well, we start with a Hadamard matrix, call it H, of order m, and m has to be
	zero mod(4), which is just a fancy way of saying that m needs to be a multiple of four. And then we then we take another matrix T, which is a matrix of plus and minus ones that is w rows by q columns.
	Bradley Jones
Bradley Jones	then we take the Kronecker product of H and T. By the way, that zero with an �X� in the middle of it is a symbol for Kronecker product and I'll explain what that is in the next slide. That operation gives us a structure for x that's m by w rows
	by m by q columns where w is less than q, so mw is less than mq. As a result, this is now a supersaturated design. We recommend that T be a Hadamard matrix with fewer than half of the rows removed. Here's an example. Here's my H. It's a Hadamard matrix.
	And one of the things about a Hadamard matrix is that the columns of a Hadamard matrix are pairwise orthogonal. If we look at
	the main effects, they're pairwise orthogonal.
	In this example, T is just H with this last row removed. And then this H cross (Kronecker Product) T
	in every element of H that's a plus one, I replace that element with T. And every element of H that's a minus one, I replace that element with -T. Now, T is a matrix. I'm replacing a single number with a three by four matrix everywhere here. H is four by four, but this new matrix here is
	12 by 16. Now I have this much
	bigger matrix that I've formed by taking the this Kronecker product of H and T.
	Of course, you don't need to do this yourself. JMP will tell you which ones you can make in the GO SSD designer and it'll all happen automatically.
	Call the Kronecker product of H and T, �X� if you look at X�X, which is what you what you look at to understand the correlations of the factors, X�X looks like this, where these are the blocks that I showed you before.
	It's block diagonal. The first group of factors is uncorrelated with all the other groups of factors. The second group of factors is uncorrelated with all the others, and so forth.
	That's a very nice property. Here again is the example. I have a Hadamard matrix that's 4x4. So, m is 4. My matrix T, what I
	was just talking about...my matrix T is 3x4, where I've just removed the last row from H and so w is three and q is 4.
	So the number of rows is m times w, or four times three, or 12. And the number of columns is m times q, which is four times four or 16.
	Now the first column is all one, so this is the call...the constant column. It's what you would use to estimate the intercept.
	The next three columns are correlated. The next four columns, columns A through D, are correlated with each other, but not with anything else. E through
	H are correlated with each other, but not with anything else; and I through L are correlated with each other, but not with anything else. You get this block diagonal correlation structure.
	Now,
	we have three groups of four factors that are correlated with each other. And then we have this first group of factors (A, B and C) that are correlated with each other more and they're also unbalanced columns. So what
	my colleagues and I recommend is that instead of actually assigning factors to A, B, and C, you leave them free.
	You don't assign them factors. And so instead of having 15 factors you only have 12 factors. Because A, B, and C are
	uncorrelated with all these other factors, we can use A, B and C to estimate the error variance. Now in supersaturated design
	There's never been a way to estimate the error variance within a supersaturated design. This is a property of group orthogonal supersaturated designs that doesn't exist anywhere else in supersaturated design land.
	Now we have three independent groups of four factors and each factor group has the rank of three. Now the three fake factor
	columns (A-C) have rank two, so I can estimate sigma squared with two degrees of freedom, assuming that that two factor interactions are negligible.
	Now notice when I created this Kronecker product,
	this group of factors is a foldover. This is a foldover. And this is a foldover. So you have three groups of factors that are all foldover designs.
	Remember that DSD was an example of a foldover design where all the factors were foldovers, but this particular structure gives you some interesting properties.
	Any two factor interactions involving from factors in the same group are orthogonal to the main effects in that group.
	I was looking at the main effects in group one, all the main effects in group one are uncorrelated with all the two factor interactions in that group. But wait, there's more. All the main effects in group one are
	uncorrelated with two factor interactions where one of the
	factors in is in group one and the other factor is in group two.
	The last thing is all the main effects in group one are uncorrelated with all the two factor interactions in any other group.
	So, this construction gives you this supersaturated design, not only is giving you a good way of estimating main effects, but it also protects you from a lot of two factor interactions.
	The only thing that you that you don't get protection from is if you have a main effect in group one and a two factor interaction involving
	main effects in two different groups, then that is not necessarily uncorrelated.
	Together all of these properties make you want to think about how to load factors into the groups. One strategy would be to say, "Well, I want to put
	all the factors that I think are most likely to be highly important into one group."
	In that case, those factors will be uncorrelated with any of the two factor interactions involving those factors. That's good. And then you would be more likely to have inactive groups and if a group is inactive, then you can pool those factor estimates into the
	estimate of sigma squared and give the estimate of sigma squared more degrees of freedom, which means that you'll have more power for detecting other effects.
	The second strategy would be put all the effects that you think are most likely to be active in different groups.
	That is, put what I think a priori is, the most active effect in group one, my second most active effect in group two, my third most active effect in group three, and so on.
	Now, if you have your most likely effects in separate groups, you're less likely to have confounding of one factor effect with another factor effect.
	Bradley Jones
Bradley Jones	My coauthors and I recommend the second strategy.
	This is a table in the paper that just got published and it shows you all of the group orthogonal supersaturated designs up to
	128 factors and 120 runs.
	Now, how do you analyze these things? Well, you can leverage the fake factors to get an unbiased assessment variance.
	You can use the group structure to identify active groups first, and then after you know which groups are active, you can do regression to identify active factors within each group.
	And as you go, you can pool the sum of squares and degrees of freedom for inactive groups and inactive factors into the estimate of sigma squared to give you more power for detecting effects.
	This is a this is a mathematical
	slide that we can just skip.
	But what the slide meant is that,
	you can maximize your power for testing each group by making your a priori guess of the direction of the effect of each factor be positive.
	Now, if you thought that the effect of a factor was negative, you would just relabel the signs of the minus ones to plus ones and the plus ones to minus ones.
	And that would maximize your power for identifying the group. What we do is we identify groups first and then we identify active factors within the groups. Of course, this how all happens automatically in the fitting.
	We're comparing our analytical method to the lasso and the Dantzig selector.
	And then we're looking at power and type one error rates using
	Go SSDs versus the standard selectors. We chose three different supersaturated designs, we, we made different numbers of groups active and we looked at
	signal to noise ratios of one to three and we had numbers of active factors per group, either one or two.
	The number of active factors can range from one to 14 in these cases.
	Looking at the graph of the power results�
	Here are the Dantzig results, and here is the Lasso followed by our two-stage method.
	Some of the powers for the Dantzig and Lasso are low. In fact, these are the cases where the signal to noise ratio is one. Otherwise, the Dantzig selector and the Lasso are doing very well.
	In facto doing as well as the two-stage method, except for these cases where the signal to noise ratio is small. However, the two-stage method is always finding all of the active factors.
	In the paper by Marley and Woods where they did a simulation study looking at Bayesian D-optimal supersaturated designs and other kinds of supersaturated designs, they basically said you...
	cannot identify more active factors than a third of the number of runs.
	Well, in our case, we have 12 factors in 12 runs, so we would expect to only be able to identify four active factors. However, in
	the case where n is 24, we identified 14 active effect, whereas n over three is only eight. You can see that we're doing a lot better than what Marley and Woods say that you should be able to do, given their simulation study. That is because of using a GO SSD.
	We did a case study to evaluate what makes JMP's custom design tool take longer to generate a design. We had
	if there are quadratic terms in the model makes it slower, if the optimality criterion is A-optimal it's slightly so; or if you do 10 times as many random starts, yeah, it's slower; if you have more factors,
	then it'll be slower. Here's our design. And here's the analysis. Let me show you
	this in JMP.
	So first, let me show you the
	design construction script.
	I'm going to...
	this is the script. m, remember, is the number of runs in the Hadamard matrix. There's a new JSL command called Hadamard. I'm going to just create a new script
	with this, and I can run the script and look at the
	Log.
	Here we go.
	Here's the new JMP 16 log. If I run the first three things in this script, you can see m is four, q is three. And here's my 4x4 Hadamard matrix. And taking the first three rows to create T and the group orthogonal supersaturated design
	is the 12x16 matrix here.
	That's the matrix and we can make a table out of it.
	And here's the table.
	So that's how easy it is to construct them by hand, but you don't have to do that. You can just get JMP to do all this for you.
	looking at the pairwise correlate column correlations, it has the same pattern that we showed you before.
	Here's the case study that we ran. We make our first three factors fake factors. Then we're going to use them to estimate the variance and then these are
	all of the real factors. When I fit
	the group orthogonal supersaturated design, there's two factors that are active in the second group. That first group I'm using to estimate variance
	One factor in the third group, and two factors in the fourth group. I have five factors in all that are active and I end up having six degrees of freedom for error.
	So that's, that's kind of an amazing thing. Let me show you how you can do this in JMP. Here's the group orthogonal supersaturated design.
	I could say I'm willing to do 12 runs. You can either do two groups of size eight or four groups of size four. This is the same design that I just ran. So that's how easy to make one of these things and then the analysis tool
	is right under the designer so you can just
	choose your factors, choose your response and go, and you get the same analysis as I got before.
	Let me wrap up by going
	back to my slides.
	When do you want to use it a Go SSD? It's when you have lots of factors, and runs are expensive, and you think that most of the factors are not active,
	but you don't know which ones are active and which are not.
	My final advice is to replace D-optimal with A-optimal for doing screening. If you were using D-Optimal before, A-optimal is better.
	Use DSDs if you have mostly continuous factors and you're interested in possible curvature
	And you don't have any restrictions about which levels of the factors you can use.
	Use Go SSDs in preference to removing a lot of factors in advance of getting any data. If you have a lot of factors and it's expensive to run the runs.

Presenter

Bradley Jones

21st Century Screening Designs (2020-US-45MP-538)

Presenter

Files