Today I'm going to talk to you about
a class of saturated mix- level main effects designs for even number of runs.
That sounds like a mouthful,
but we'll get an understanding of what that is by the time we're finished.
If you see in my JMP journal here,
here's a rough idea as to what the outline is going to look like.
Now, at the beginning of though, I do have to call out,
so I have three collaborators on this project.
Bradley Jones, who should be familiar to many of you at JMP,
Dibyen Majumdar and Chris Nachtsheim.
Just some history and preliminaries .
I'll say usually...
If you see in there, we talked about main effects designs.
Now, usually, when we think of screening designs,
we often think of factors being all at two levels.
Your standard fractional factorial designs that you see in textbooks,
but really there's this big question, what about nonlinear effects?
Of course, it's great we want to find out those most important main effects,
but what happens if there is some nonlinearity?
In particular, in 2011,
Definitive Screening Designs or DSDs hit the scene.
The big thing with those DSDs,
here we were assuming all the factors were continuous.
Each factor and DSDs are going to be at three levels.
That gave some hope of being able to detect quadratic effects
when they were large.
I'll say that was one of the big popularity of DSDs.
The designs we're going to talk about here.
Our main effect, if you think of DSDs,
not only were they good at picking up main effects,
we're also looking at quadratics and interactions.
The designs we're looking at here
are really main effects screening as our primary goal.
By main effects screening, I'm saying we have our list of factors
and we want to find out which of those are the primary drivers.
We want to find out those significant main effects.
If we're really lucky, we may get some quadratic interactions.
But again, main effects screening is the big thing.
What you're going to find here is we're going to have a mix
of three- level and two- level factors.
Whereas the DSDs had everything at three- levels,
here we have more of this mix of three and two- levels.
One of the things to pay attention to
is just like definitive screening designs.
When we're talking about these three-level factors,
we're going to assume that they're continuous.
In particular, that means we're not looking for balance.
Often you see in that title where we talk about mixed levels.
We're talking about three and two-level factors.
But I'll say traditionally when we think of orthogonal rays,
the three-level factors are categorical,
and so we want to see each level an equal number of times.
What we're going to find here is that these three-level factors
are j ust going to have a few zeros.
One of the other big things, though,
is that we're going to have almost as many factors as we do runs.
That's where that idea of saturated and the title came from.
Saturated just means effectively that I have just as many factors as I do runs.
That's where also that main effects screening
because now I have so many factors,
if you start considering quadratics interactions,
that's an awful lot of terms.
Because we're looking at so many factors,
we're just hoping we can detect those significant main effects.
If you think about other designs,
if you think about this mix of three- level and two levels,
now immediately what might come to mind
is some of the classical Taguchi designs, so the L18 or the L36.
Those of you who are familiar with JMP,
you may know that you can create Definitive Screening Design
but added two-level factors.
The designs here I would almost look at as an extension to some of those.
Here we're going to have a lot more two-level factors
than you might want to, though, in our standard DSDs.
I'll say this is also an area, though, that's picked up in steam
a lot in these past few years.
You see some references here
as to other authors that are thinking about this same problem,
including a paper that I'll come back to at the end.
You see that last, the Jones, Lekivetz, and Nachshteim paper.
That's actually related to this work as well.
Let's see, hopefully we're all in the same.
If we go back to that title.
The saturated just means we have a lot of factors relative to the run size.
Our mixed level means that we have some two-level and three-level.
Just to mention the two- level could be continuous as well,
where we're just not interested in looking at those quadratic effects.
Main effects design says that's where
our most important thing is finding those significant main effects
and that even numbers of runs,
it turns out the designs, if you remember that recall the outline,
all these designs we're looking at are going to be an even number of runs.
Now, when we think about building these designs,
so in some of these preliminary,
I should really talk about the building blocks.
These designs are going to be built upon
using other matrices or other designs in the literature.
Depending on how familiar you are with Definitive Screening Designs,
there's this idea of something called a conference matrix.
A conference matrix is just an M by M matrix
that we use to construct Definitive Screening Designs.
One of the nice things with conference matrices
is that in general,
they exist for multiples of two rows and columns.
I said in general,
there's a conference matrix for every multiple of two
from 2 to 30, except for 22.
Something to mention here, though,
is that in the cases where a conference matrix does not exist,
or let's say if you have an odd number of runs,
you can use something called a pseudo- conference matrix.
A pseudo- conference matrix says,
I want to make it look as close as I can to a conference matrix,
which has this special property.
What properties a conference matrix have?
Well, if I take C transpose T, or conversely, C C transpose,
it's going to be M minus 1 times the identity matrix.
What that really means is that these columns,
the columns of the matrix C, orthogonal.
If I take that cross- product of any two columns,
we're going to get a zero.
But notice we have that M minus 1,
because one of the other features of the conference matrix
is that each row and column has exactly one zero.
If you have JMP 17.2,
so I'll admit that this JSL first shows up in JMP 17.2,
so we have this command for conference matrix.
Here I can just ask for the same 6x6 conference matrix.
Again, what's the special property?
Well, if we look, so if I take C times C transpose or conversely,
I get 6 minus 1, which is 5 times the identity matrix.
Each of those columns is going to be orthogonal.
Again, starting a JMP 17.2, you can create your own conference matrix
just by giving it the order.
Here I wanted a six-run conference matrix, so I was able to put that in.
Let's try it with eight as you'll see.
There's an 8 by 8 conference matrix.
That's one of the building blocks that we need.
Another one is a similar type of structure.
It's called a Hadamard Matrix.
The difference is a conference matrix had values of negative one, zero, and one.
A Hadamard matrix just has values of plus and minus one.
A Hadamard matrix exists for most multiples of four.
In particular, this is where when you hear Hadamard matrix,
another thing you'll hear about often times is an orthogonal array.
In particular for a Hadamard matrix, it has that same property.
If I take it by its transpose,
in this case, I get N times the identity matrix.
If you recall the conference matrix,
it was that M minus 1,
the order because we had one zero in each row and column.
Here it's going to be N times the identity matrix.
The idea here is that if I take any pair of columns,
we have that concept of orthogonality for any pair of columns.
Similar to the conference matrix,
we actually have a special command in JSL for constructing a Hadamard.
Let's take a look here.
If you notice, let's just say Hadamard 8,
this is going to give me an 8-run Hadamard matrix.
If we see about taking a Hadamard,
the strand rows, there we get eight times the identity matrix.
Again, so what that means, if I take any pair of columns,
they're going to be orthogonal, which you can actually already see.
Just pretend that this first one is like an intercept.
You see, all of these are balanced.
I'm going to get that idea of orthogonality.
We're actually almost there with our building blocks.
The last piece that we need is something called the Kronecker product.
What you'll see is that throughout we may not even really need
to think of it in terms of a Kronecker product.
Just often when we create these designs, that's the way we like to think of these.
You'll see a Kronecker product is denoted by this symbol.
It looks like a multiplication with a circle around it.
In JMP or another definition for Kronecker product
is called the direct product.
It just happened it's a convenient way to construct designs.
All of this, Kronecker product or direct product is,
if I take a matrix A and the Kronecker product of B,
I take each element of A
and apply it to the entire matrix B over and over again.
Where this comes in handy,
let's see where you may have seen something like this before.
Again, we have a direct product, a JSL command.
Let's say if I just wanted to start with that 6x6 conference matrix.
Now let's say if my matrix A, so think here in this Kronecker product A,
I just take these 2 by 1,
so two rows plus one and a minus one,
and direct product with C.
What do I get here?
Well, this is effectively if I added a center run,
this would be a 13-run DSD and six factors.
By taking this one and minus one,
so with the Kronecker product up here with this plus one,
I get that conference matrix C, and down below I get negative C.
This Kronecker product is just a convenient way
to think about things like what we might call a f oldover.
With those preliminary is done,
now we can actually start talking about these different constructions.
The first method is the most, I'll say the nicest of all of them.
What we're going to say, this is where our run size
is going to be a multiple of eight.
What are we doing here?
Well, here we're going to start.
We're going to have a conference matrix of order M equals 4K.
Again, remember, the conference matrices
tend to exist as long as we have an even number.
Likewise, for the Hadamard matrix,
that's where we're looking for multiples of four.
Here I'm going to take a conference matrix
of order M and a Hadamard matrix of the same order
where we're assuming here that both exist.
What I'm doing here is I'm going to fold over.
You can actually express this as a Kronecker product,
but here I find it more convenient just to write it this way.
You see, this first part looks like a Definitive Screening Design.
I'm taking a conference matrix and folding it over.
Then on the other side, I'm replicating this Hadamard matrix.
I'm taking a copy of that.
What do we get with this design?
Well, these first M columns that are formed from the conference matrix,
that's going to give us M three-level factors.
The remaining M minus 1 are all going to be two-level factors.
This C part is going to be for three-level,
this H part for two-level.
What do we end up with?
Well, what we did, we basically doubled these here.
We're going to have two M-runs and two M minus 1 factors.
Let's take a look at what this might look like.
Let's see an example here.
Let's take my C.
I'm going to just create an 8 by 8 conference matrix.
First, let's construct these three-level columns.
I'm going to take that direct product.
Again, in this case, I want that foldover, I want C and minus C.
Let's take a look at what C looks like.
Again, this is just that foldover structure on that.
One thing here, you noticed I said the remaining M-1 column
for the two-level factors.
The reason for that...
Let's take a look at the Hadamard matrix of Order 8.
If I'm going to replicate this, if I'm going to copy this,
well, this first column here is going to be for the intercept.
I don't want to put that as one of my design factors
if it never changes,
if it's constant throughout the entire thing.
If you notice here, I'm going to just drop the intercept.
Now I have an 8 by 7 design.
If you notice, I'm going to use the direct product again here.
But instead of with the conference matrix
where I was using one and minus one, I just want to make a copy
of that Hadamard matrix H without the intercept.
Let's give that a look.
We can take a look here on this matrix
where you see that the one, one, one.
You can actually see where it gets just copy it again.
It's just the same matrix stacked on top of itself.
If I concatenate all of those together, you can see I have a 16 by 15.
I can actually just create that data table.
I have this design now, a 16-run design with 15 factors.
The first eight of those are three level and the remaining seven are at two levels.
Let's just take a look.
Let's go to design diagnostics
and let's see what this looks like and evaluate design.
You can see I just created a main effects.
This might be hard to see, so this looks pretty messy,
but you can already see there's a special structure with these designs.
One thing I want to point out, let's get rid of the alias terms.
Let's just look at the correlations here with these main effects.
You can see in this case,
actually, all my main effects are orthogonal to each other.
One thing to point out here,
because I was using that Hadamard matrix as a building block,
those two-level designs, we have all that nice orthogonality there.
You'll notice this fractional increase in confidence interval length
is a little bit higher.
Why is that?
Well, that's because we have these three-level factors.
Those first factors are at three levels.
What that means is that if there is a quadratic effect,
now this is giving me some hope of detecting that.
Again, don't forget, we're already at 16 runs and 15 main effects.
If we start thinking about quadratic effects and interactions,
we really have to hope for those large effects
when it comes to doing any model selection.
If you think of a traditional design where we only had two levels,
in that case, we would have no hope
of being able to detect any quadratic effect.
That's that first construction.
In some sense, that's where everything works out nicely.
I have a conference matrix and I had a run matrix available to me.
That gives me a run size that's a multiple of eight.
Now what happens if I don't have one of those?
Let's say my design is going to be a multiple of four.
The run size is going to be a multiple of four.
Let's assume now that I have a conference matrix available to me,
but maybe I don't have a Hadamard matrix.
If you recall before,
where for my Hadamard, I need it to be a multiple of four,
the original run size, so that when I doubled it, it was a multiple of eight.
Instead, maybe if I don't have a Hadamard matrix,
I could use something like a D-optimal main effects plan.
This would just be if I went into custom design.
Let's say if I wanted for six runs, I would go into custom design
and say I have five main effects, five factors,
and I want six runs for a main effects.
This construction actually looks a lot like it did in method 1.
The only real difference is instead of that Hadamard matrix,
now I'm going to be using this D-optimal main effects plan.
But it turns out to be the same thing here.
I'm going to have those first M columns for the three-level factors,
the remaining M minus 1 for the two-level.
I'm really at that idea of saturation
because by the time I factor in the intercept,
I'm at two M-runs and 2 M minus 1 factors.
Let's take a look at how this one might look.
In this case,
I'm going to go with a 10-run conference matrix,
which again, because that's a multiple of two,
I can create a conference matrix.
The one thing to pay attention to here, though,
because we're at 10- runs,
there's not going to be a Hadamard matrix of order 10 available to us.
Let's construct though first though, let's just fold over that CC.
Again, that's just that DSD-like structure.
Now in this case, because I don't have that command,
and so this is where you could decide how to do this.
You may actually even have your own design that you want to use in this case.
I'll say in this case, I'm showing a D-optimal main effects.
You may want to use something l ike a Bayesian D-optimal
where two-factor interactions are if possible.
But so in this case, all that I've done
is I've taken the D-optimal, 10-run design for nine factors,
and I've just happened to include the intercept here.
If you go into Custom Design, this is what you get from the model matrix.
The model matrix actually includes the intercept by default.
That's in fact where I had this from here.
Again, I'm going to drop.
Let's drop that intercept column.
Again, here I'm just replicating the exact same thing again.
I've just folded over, not folded over.
I've just replicated the exact same thing twice.
It'd be hard to see as to where it was replicated here.
But again, all that we've done is just made a copy.
I made a copy of that doptN10m9 twice, concatenate those together.
Let's create the data table again.
Again, 20- runs and 19 factors.
Again, keep in mind that my first 10 factors
are all at three levels, the remaining at two.
Let's take a look at evaluate design again here.
I'm going to put all of these factors in.
In this case, before,
let's go directly to remove those alias terms.
Let's take a look at the color map.
This still looks like a pretty good color map to me.
What do we notice here versus the last one?
One of the biggest differences is,
well, here our three levels are still going to be orthogonal,
and that's because we were using a conference matrix.
The three- levels and the two- levels are orthogonal.
In particular, my three-level factors are orthogonal to the two-level factors.
But because I was using that 10-run design,
it turns out we can't get perfect orthogonality
for the 10-run design with nine factors.
If we look among those two-level factors, we have some small correlations there.
One thing to point out,
there does exist a Hadamard matrix of order 20.
In some sense, we are taking a little bit of a hit on those two-level factors.
But if you take a look, the cost of using this type of design
versus, let's say, everything at two levels,
we have about a 5% increase in the confidence interval length.
But the nice thing here now is that now we have those factors at three levels.
If we really are worried about quadratic effects
appearing in these first ones, now we have a chance of detecting those
at a small price to that estimation efficiency for those main effects.
That was great in the case
that we still had a conference matrix that existed, but no Hadamard.
This last method,
this is where we don't really fit into either of those cases.
In this case now,
we're talking about a run size is going to be a multiple of two.
In this case, what we have is we don't have a conference matrix available to us.
This is where we're going to use that pseudo- conference matrix.
I'll say if you're really interested in that,
I'll go back to these preliminary.
In the...
Let's see.
Actually, it was the original DSD design.
Sorry, this 2011 Definitive Screening Design.
If you also look, I have it in the list of references at the end.
Let's see, those who are particularly interested.
This original class of three-level design for Definitive Screening
in the presence of second- order effects.
This paper was written before they were aware of the existence
of this idea of conference matrices.
In that paper, they talk about a general purpose algorithm
for creating something that looks like a conference matrix.
You set zeros along the diagonal,
and then the rest of the values are going to be plus and minus one,
where you're trying to make a main effects D-optimal design.
These pseudo- conference matrices, as they were,
they look like a conference matrix.
You can use that algorithm when the conference matrix doesn't exist.
Really, what it's trying to do is to drive it to look as close as it can.
If you remember a conference matrix,
if you take C transpose, you get zeros and the off diagonal
for perfect orthogonality,
it's going to be trying to drive it to look like that
where I can't make it perfectly.
Similar to the case we had for method 2,
our T is going to be a D-optimal main effects plan.
What do we end up with here?
Again, our first M columns are still going to be three-level factors,
the remaining M minus 1 for the two- level.
Again, we're still at that case of saturation
by the time we factor in the intercept.
The cost here is that we're not going to get the nice orthogonality
that we may have had in method 1 and 2.
Let's take a look at how this one works here.
My C in this case,
and I'll do that CC.
What I started with here
was a pseudo- conference matrix of order nine.
Nine is not a multiple of two.
I have to do something that looked like a pseudo- conference matrix.
Let's just take a look here.
Let's actually take a look and see what that C transport C looks like.
You notice I can't get that perfect orthogonality,
but instead I have eight along the diagonal
and then these plus and minus ones on the off diagonals.
We're close.
I have that CC that was just doing the same thing, folding that matrix over.
Likewise, so T, that's going to be of the same order.
I took the nine-run, eight-factor, D-optimal design.
In this case, I've actually already removed the intercept.
We're just going to replicate that design again.
I have this 18 by 8.
I'm going to combine those two pieces together.
Let's take a look at the table.
This is a particularly difficult design to generate in general.
In this case, I don't have a nice number.
My run size is 18.
Now I have these first nine factors at three levels
and the remaining eight at two levels.
Not surprisingly, there is a cost of this.
Again, let's just remove those alias terms again
and take a look at our color map.
You see now, my two and three levels are actually orthogonal to each other,
but the three levels have a small correlation among them.
That was where, remember, if you recall,
we saw that plus and minus one and that off diagonal.
Likewise for the two levels.
We actually have one quite high correlation here,
but in general, so about 0.1.
In this case, we don't even have an orthogonal design to compare it to.
We actually have a couple of factors that we may be worried about a bit,
have a little bit of a larger fractional increase.
Those three levels will say, well, we can't get those to be perfect as it is.
This fractional increase is compared to the hypothetical orthogonal design,
the orthogonal array, which doesn't even exist.
We're paying a small price, but we still have some generally nice properties.
The three levels are orthogonal to the two levels,
and we've minimized that correlation in general.
In some sense, those are those three methods.
The nice thing is depending on the run size that you have
and depending on the number of factors, this methodology or this class of designs
really gives you some flexibility in the number of runs.
Of course, if you can afford a multiple of eight or a multiple of four,
the properties are going to look a lot nicer.
But I mean, when runs are really expensive,
we still have this method 3,
which is going to give you a reasonable design.
With that, I just want to give some final thoughts here.
I have a link.
If you take a look at this journal afterwards,
I have a link to a presentation from JMP Discovery Europe
from Bradley Jones that was on orthogonal mix level designs.
The designs in this presentation, the orthogonal mix level designs,
will almost look very similar to what was presented here.
The designs presented in this presentation are for when you're closer to saturation.
I would say these orthogonal mix-level designs
from the previous discovery, and we actually have a paper on that.
That's this Jones Lekivetz, and Nachtsheim
that was in JQT, Journal of Quality Technology.
Those designs work very well
when you have about half the number of factors relative to the run size,
which sounds a lot like DSDs,
but it's when you have more two-level factors.
In the design presented in that presentation,
you can go up to about three-quarters of the run size.
The designs presented here
are really when you're closer to that saturation,
where you say, well, no, I really have a lot of factors
that I'm interested in, and the runs are really expensive.
The designs here to fill that last gap when you want to get close to saturation.
I showed you these designs look nice,
but can they actually do anything when it comes to model selection?
I'll say here.
Again, we have run a lot of simulations on these.
This isn't doing it properly, this is just for a single realization.
Let's say here
my factors, so call one, call five, call 13 and 14,
I've chosen it as being significant.
In base JMP, you might want to use something like stepwise.
Let's take a look.
Here I've done a reasonable job.
I may have ended up with one extra factor,
but in some sense, I'd still be
quite happy that I've been able to pick up those effects there.
Let's try this same thing with generalized regression.
If we take a look, the same thing.
Let's see, 1, 5, 13, and 14.
If we go back, 1, 5, 13, and 14.
In this case,
both of you have JMP Pro with generalized regression or Fit stepwise,
both were able to pick up those main effects that we had in there.
Just to show you now, if you have a large quadratic effect,
I'll say because compared to DSDs where we had that center point run,
this we only have two zeros.
We don't even have a third for a center point
and we have a lot more factors.
Detecting quadratic effects is still going to be difficult.
But let's just pretend we have a large one.
Here I have again, the 1, 5, 13, and 14.
But now I've also added a quite large quadratic effect for the column one.
Let's just take a look here.
Let's try to fit stepwise again.
Now I've actually just added those quadratics in the model.
Again, I have one extra term,
but if you see, even in this case, it did actually pick up that quadratic effect.
Not only did it pick up the correct main effects,
it did detect the quadratic effect that I had.
Let's try the same thing.
Let's see the model launch.
It looks like I already have it shown here.
The same thing.
You see the 1, 5, 13, 14,
and generalized regression was also able to pick up that quadratic effect.
I'll say I would not expect to anticipate
that you can detect that many quadratic effects.
But even if you look at your residual plots or your main effects plots,
even those two zeros give you some indication
that maybe I do want to follow up
and take a look at those factors a little bit deeper for quadratic effects.
Where again, in your traditional screening,
if you're only doing things at two- level, you wouldn't have the chance to do that.
With that, again, I will post this journal where this video is located,
but I'll also flash up these references at this time.
With that, thank you for taking the time to watch this video,
and please share any messages you have in the community below.