DOE Gumbo: How Hybrid and Augmenting Designs Can Lead To More Effective Design Choices (2020-US-EPO-568)
Heath Rushing, Principal, Adsurgo
DOE Gumbo: How Hybrid and Augmenting Designs Can Lead to More Effective Design Choices
When my grandmother made gumbo, she never seemed to even follow her own recipe. When I questioned her about his, she told me, “Always try something different. Ya never know if you can make better gumbo unless you try something new!” This is the same with design of experiments. Too many times, we choose the same designs we’ve used in the past, unable to try something new in our gumbo. We can construct a hybrid of different types of designs or augment the original, optimal design with points using a different criterion. We can then use this for comparison to our original design choice. These approaches can lead to designs that allow you to either add relevant constraints (and/or factors) you did not think were possible or have unique design characteristics that you may not have considered in the past. This talk will present multiple design choices: a hybrid mixture-space filling design, an optimal design augmented using pre-existing, required design points, and an optimal design constructed by augmenting a D-optimal design with both I- and A-optimal design points. Furthermore, this talk presents the motivation for choosing these design alternatives as well as design choices that have been useful in practice.
Auto-generated transcript...
Speaker | Transcript |
| Heath Rushing | My name is Heath Rushing. I am a principal for Adsurgo and we're a we're a training and consulting company that works with a lot of different companies. |
| This morning, I'm going to talk about some experiences that I had working with pharmaceutical and biopharmaceutical companies. | |
| A lot of scientists and engineers are doing things like process and product development and characterization formulation optimization. | |
| And what I found is is is a lot of these...a lot of these scientists had designs that they use in the past with a similar product or process or formulation. | |
| And what they did is is going forward, they just said, "Hey, let me just take that design that I've used in the past. | |
| It worked. You know, it worked well enough in the past. So let's just go ahead and use that design." | |
| In each of these instances, what we did is we took the original design and we came up with some sort of mechanism for doing something a little different. Right. We either augmented it with a | |
| with a with a different sort of optimization criteria or we augmented it before they added runs or after they added runs. | |
| In the first case is what we did is, is was we built a hybrid design. Right. And then the first case was a product formulazation...I'm sorry... a formulation optimization problem, | |
| where a scientist in the past was run...had a 30 run Scheffe-Cubic mixture design. In a mixture design, | |
| the process parameters are variables are factors in the experiment are mixtures. And then, so there is certain percentage where the overall mixture adds up to 100%. | |
| Right, they they felt this work well enough and help them to find an optimal setting for the for the formulation. However, one thing that they really wanted to touch more on is, they said, you know, | |
| these designs tended to to to look at design points in our experiment near the edges. And what we want to do is is further characterize the design space. | |
| So we took the original 30 run design, and instead of doing that, what we did is we run a we...we developed an experiment constructing the experiment where we ran 18 | |
| mixture experiments and then we augmented it with 12 space filling design. And a space filling design is, it's used a lot in computer simulations. | |
| And really, you know, I said this at a conference one time, I said, "You know it's used to fill space." But really what these designs do, and I'm going to pull up the the the comparison of the two, | |
| is it's going to put design points. In this one, I try to minimize the distance between each of the design points. | |
| As you see as the design on the left, the, the one that they thought was well enough or was adequate was the 30 run mixture design. And as you see, it operates a lot near the edges and right in the center of the design. The one on the right was really | |
| 18 mixture design points augmented with 12 space filling design points. So it's really a hybrid design, it's really a hybrid of a mixture design and a space filling design. | |
| As you can see, you know, based upon their objective trying to characterize | |
| that design space a little bit better, as you can see, the one on the right did a much better job of characterizing that design space, right? It had adequate prediction variance. It was a it was a design they chose to run and they found a and they found their optimal solution off of this | |
| The second design choice | |
| was, and this is used a lot, in a process characterization is, back in the old days back before a lot of people used design of experiments in terms of process characterization, | |
| what a lot of scientists would do was, was they would run center point runs like its set point | |
| and then also do what are called PAR runs, or proven acceptable range, right. So say that they had four process parameters. What they would do is is they would keep three of the process parameters at the set point and have the fourth | |
| go to the extremes. The lowest value and the highest value. And they would do it for each...they would do a set of experiments like that for each the process parameters. | |
| What they're really showing is that, you know, if everything's at set point, and one of these deviate near the edges, | |
| then we're just going to prove that it's well within specification. Right. And then so they still like to do a lot of these runs. | |
| The design that I started off with was, I had a had a scientist that took those PAR and those centerpoint runs and they added them after they built an I optimal design. | |
| And I optimal design is used for for for prediction and optimization. And in this case is is that's the kind of design that they wanted, but they added them after the I optimal design. | |
| My question to them was this, why don't you just take those runs and add them before you built I optimal design? If that was the case, | |
| the ??? algorithm in JMP would say, "You know, I'm going to take those points and I'm going to come up with the, the next best set of runs." | |
| Right. So we took those 18 design points and we augmented them with with 11 more...I'm sorry, the 11 to...the original 11 design points and 18 I optimal points. Whenever we did this, if you look in the design, the, the, this is where the PAR runs were added... | |
| were added prior to, and you see that the power of the main effects, in factor interactions, the quadratic effects are higher than if you added the PAR runs after. You see that the production variance, | |
| if you, if you look at the prediction variance is, the prediction variance is very similar. But you see, is like right near the edge of the design spaces, you see that those | |
| PAR runs, whenever we had the PAR runs augmented with I optimal, were a lot smaller. | |
| The key here is is whenever I was looking at the correlation is I think the correlation, especially with the main effects are a lot better with with the PAR augmenting and two I optimal versus what they did before, where they took the I optimal and just augmented those with the PAR runs. | |
| The third design. The third design was was was when I had a scientist take a 17 run D optimal design and they augment it with eight runs and went from a D to an I optimal design. | |
| Now they started off with D optimal design, a screening design, they augmented it with points to move to an I optimal design. JMP | |
| has a has a...it's not a really a new design, but it's new design for JMP; it's called A optimal design. And A optimal design | |
| allows you to to weight any of those factors. Right. And so I had an idea. I just said, "You know, I have many times in the past, went from a D | |
| augmented to an I optimal design. What if we did this? Really, what if we took that original 17 run D optimal design and augmented it to an I, then an A, where we weighted those quadratic terms, | |
| Or we took the D optimal design, augmented it to an A optimal design where we where we weighted the quadratic terms and then to an I optimal design?" So it's really two different augmentations, going from a D to an A to an I, | |
| and D to an I to an A. Also went to straight D to A. Right. And I wanted to compare it to the original design choice, which was | |
| a D versus an I optimal design. Now, I really would like to tell you that my idea worked. But I think as a good statistician, I should tell you that I don't think it was so. | |
| If I look at the prediction variance, which, in terms of response surface design, we're trying to minimize the prediction variance across the design region, | |
| is you see the prediction variance for their original design is is lower. Okay, even even much lower than whenever I did the A optimal design, just straight to the A optimal design. | |
| If you look at the fraction of design space, you'll see that the prediction variance is much smaller across the design space than the than the A optimal design and it's a little bit better than when I went from D to A to I, | |
| and D to I to A. The only negative that I saw with the original design compared to the other design choices was, you know, there was there was some quadratic effects, right, there were some quadratic effects | |
| that had a little bit of higher correlation, little bit higher correlation than I would like to see. And and you see what the A optimal design, it has much lower quadratic effects. So my my original thesis | |
| many times, scientists and engineers have designs they've done in the past. | |
| And I always say is, it makes sense that we just don't want to do that same design that we've done in the past. Let's try something different. | |
| The product can be a little bit different. The process can be a little bit different. The formulation can be a little bit different. If you use that to compare to the original design is you can pick your best design choice. | |
| I would like to, you know, last thing I would like to thank my my team members at Adsurgo. | |
| We always have, you know, team members and also our customers...our customers coming up with challenging problems and our team members for always working for for optimal solutions for our customers. | |
| Now, last thing that I have to do is, is these these designs were really, really taken from examples from customers, but they weren't the exact examples. There's nothing with their data. | |
| So I would like to give a give a shout out to one of my customers Sean Essex from Poseida Therapeutics that often comes up with some very hard problems and sometimes | |
| he'll come up with a problem. And I'll say, you know, this is this is a solution and it's something that we really haven't even seen yet. So | |
| have a great day. |
