Optimizing Biomanufacturing 1,2,4-butanetriol with JMP Hybrid DOE and SVEM (2026-US-PO-2976)

Hello, I'm Jay Davies. I'm with the US Army Combat Capabilities Development Command, the Chemical Biological Center, and we are in Aberdeen, Maryland. I'm going to talk today about a particular design of experiment that we did to optimize a cell-free pathway for biomanufacture of a chemical, which I'll call BT. It's 1,2,4-butanetriol. Where we are at the DEVCOM CBC, we do research, everything from basic research to development of optimization of fielded systems, and pretty much everything in between.

Personally, I've been here about 20 years. I've been using JMP about 20 years, and I guess maybe about 25 years, actually. They would call me a statistician. I do experimental design. I do different forms of statistical analysis, but I'm probably comparable to an in-house consultant in that I live in the Decontamination Sciences Branch where we develop decontamination for chemical biological warfare weapons.

Some of my work, some of my DOEs are along those lines, but I also work with groups that are doing biological; this one, for example, was a biomanufacturing project. I work with 3D print people, work with people that are synthesizing DNA. I've done work on the smoke grenades. I've done work in all kinds of fields. The experimental design has let me, as George Box would say, play in a lot of people's different backyards, or maybe it's Tukey that said that.

Here we go with today's discussion. This project will deal with optimizing a cell-free pathway for cost-effective biomanufacturing of the chemical BT. I'm going to be using JMP Pro. I was probably using 18 when we did this. I'm using 19 now. There are some features, namely the SVEM, Self-Validating Ensemble Model, technique that's exclusive to JMP Pro. Some of the other models I build, the space-filling technique, I'm not sure that all those capabilities are in regular JMP. I use Pro also for functional data analysis and quite a few different things.

Some acknowledgments for this work that I'm going to talk about. This was very much a team effort. There's a paper that's currently in review. It's listed here, and I've listed all the co-authors that worked on this. There were four of us at the DEVCOM Chemical Biological Center: Katherine Rhea, Marilyn Lee, David Garcia, and Matthew Lux. Garcia and Lux have since moved on to different jobs. We also had several, as you can see, university partners partnering with us.

Basically, everybody in that group, those are the biologists. I'm the statistician, I'm the experimentalist. My involvement was at the experimental design level, the statistics, building the designs, interpreting the results, optimizing the systems. I'm by no means a biologist. I will attempt to give you some background and some information into the biology just to give some context as to what we did.

As an introduction, I mentioned this is a project to improve the biomanufacturing of BT. It's a very important precursor for a variety of applications, a lot of things that the Department of War and the Army are interested in, and industry. Efforts to produce BT through engineered cells have had mixed success, taken a lot of resources, and they're not super productive.

The angle that this group is taking, we are attempting to use a cell-free metabolic engineering technique. This is where we lysate the cells, which means pull the extracts out of them, take them out of the shell, out of the cell itself, and then work with these biomechanisms independently. This is going to involve a formulation DOE. There's a precursor, xylose. This group developed, the biologists developed an enzyme mix, and then we're going to mix it with some other cofactors.

The cell-free expression systems, as I said, is we take cells, we harvest the insides, the biological pieces, and we work with those rather than working with the cells themselves. I think the advantages of that is the shelf life is not as limited, and there's a lot more things you can do by mixing, matching, and blending the lysates or the cell-free expressions rather than working with the cells themselves.

The bottom line up front: Why did we do this project? We wanted to find what's the optimal formulation that maximizes the production of the BT while at the same time minimizing cost. Production and cost, both are important. Maximize production, minimize cost.

The methods and objectives. I'll circle this work in the enzymes themselves. This was all done just by the biologists. I wasn't involved with this part. They did a lot of, what is it, the design, build, test, learn cycle to tune and mix and match these enzymes to come up with a cocktail that had the potential to be very productive.

Once that was completed, that's about where I came into the project. The question was, well, if we have this enzyme mix, we know we need to put the xylose in because that's the precursor to make the BT. Then we have these four other components I'm showing over here on the right. These are salts and cofactors. It's these six components.

The question is, we've already tuned, or the biologists have already tuned this enzyme mix. They've hand-selected some of these, but it's not clear as to what the actual blend is or what the optimal blend might be. That's where I got involved with the experimental design part of it.

This becomes a six-component formulation design. The six components is not... It's probably moderate. We've done here formulation designs with 10, 20, 30 components, and we've done some with three and four. Six is... It's moderate. But the catch is here: With the subject matter expertise that we have from the biological team, we know that there's likely going to be complex nonlinear blending effects, and they could be two ways, they could be three ways. This is going to be a cubic model for this DOE. It makes it a little more challenging.

Now, I said that the objective was to find the optimal formulations that maximize BT production while minimizing cost. The key caveat to that is we want to do it in an experimentally efficient manner, which means that we don't want to do something like this, what I call unstructured experimentation.

That has a place, and not that it can never work and never be productive, but it's usually not the most efficient way to characterize a system. Even if you leverage your subject matter expertise, if you find a result that meets the goal, you don't really know if you've thoroughly explored the design space, and it doesn't generally yield a predictive model.

We want to, I'm illustrating it with the tangled-up Christmas lights here is, in the worst case, you end up collecting a lot of data that's confounded, that's disjointed, that you can't get a lot of information out of. What we want to do, and we decided to do, is we're going to use DOE, which is more structured, and it's compact.

The methods that we use to do the experimental design is, again, at this time, I was JMP Pro 18, and I'm going to build a hybrid design. The hybrid design combines, in this case, an I-Optimal design with a space-filling design. The reason for introducing the space-filling points is we know that this system is six components, not overly complex, but it has some complexity, but we have inside information that we're going to see nonlinear blending effects.

They might be two-way, they might be three-way. As of recent results that I've seen, people are having good results with these space-filling designs because where I-Optimal tends to cluster points in the center and the edges and the corners, the space-filling puts some more points on the interior. That's what we did here. We used JMP to generate 25 space-filling points using the Fast Flexible Filling routine. Then we took those points, and we augmented the design using the Augment tool in JMP to add 30 I-Optimal points.

At this time, we have 55 DOE sample blends. I also call them points. Those are the sample blends. There are no exact replicates. These are 55 unique blends.

Here is our design space. These are our six components. They have a letter code here for simplicity. Our constraint was each sample has to add up to a volume of 10 microliters, and five of the components can go down to zero. That means they can be not in the blend, and they can be as much as 10% in the blend. The enzyme mix will always be there. It can never go less than 46% in the blend.

The model that we decided to start our DOE, to build our DOE around is a Scheffé Special Cubic model, which is shown here on the right. You have the linear blending terms, two-way nonlinear blending terms, and all of the possible three-way nonlinear blending terms for a total of 41 terms that could possibly be in the finished model, but we're going to use the 41 to create the starting model.

If you look at this ternary plot, these are the I-Optimal points that are in our design. It's a ternary plot, so I'm showing it relative to three of the components, and it doesn't matter which three we happen to be looking at. You get the idea. The I-Optimal points tend to be the corners, the sides, and a couple centered around the middle here.

If we look at this shaded area, which is the midway all the way around, not a lot of I-Optimal points, maybe one falls in there. But if we add on to it our 25 space-filling points, if I turn those on, now you see the blue points, the space-filling points, they cover a lot of that interior. We have confidence that this is going to give us better ability to predict if we have highly complex two-way and three-way nonlinear blending effects, and we're building in the SVEM, Self-Validating Ensemble Model, as a tool that takes advantage of those space-filling points.

What results did we get? We started off by saying, let's not do the SVEM model now. Let's fit this as if it was a normal regression model, ordinary least squares regression, fitting the same model, the 41-term model. We used a step regression, forward step with the AIC as a stopping criteria. JMP told us that we ended up on this 18-term model.

Over here on the left, you see the actual BT production versus the model predicted for the 18-term model. BT is in millimolar. I guess we're actually talking about the titers. When I say production, I'm talking about the titer levels. We want the titer level to be as high as possible.

If you look at this plot, you'll see I've got these blue triangles, and there should be six of them. There's a couple on top of each other down here. Those are checkpoints. Years and years ago, when I was working with Dr. Tom Donnelly, he used to work here at, it used to be ECBC, Edgewood Chemical Biological Center, back in there, he started me with the idea of using these checkpoints. What they are is, if you remember, I said we had 55 points in this model. Six of them were held back.

We ran them, but we held that data back when we fit the model. Then once the model's fit, optimized, and we think it's a good predictor, then we let it try to predict those points.

You'll notice here, roughly half of them or most of them are predicted pretty well. This one is off a little bit. We do have a pretty big miss on this one checkpoint here, which is something of a concern because the idea is if the points weren't used to fit the model and the model can predict them, then the model is not overfit. In this case, we're doing a pretty good job on the checkpoints aside from this one point. This model is leaning like it might have a little bit of overfitting going on, but we're not sure. I've marked here it's sample 41 where we had the fairly large miss.

Then we moved over to JMP Pro and fit the SVEM technique. For those not familiar with the SVEM technique, it uses a random weighting scheme so that all of the data... Normally, when you have a machine learning method, you have 10,000 points, you're going to hold back 30% as your validation points and train on the rest.

But when you only have 55, if I hold back 30%, I'm killing my DOE. I don't have enough points to do that. The SVEM technique is very nice here because it's going to use all the data for training and all the data for validation, which doesn't seem to make sense, except it does that by using a randomized weighting scheme.

Point number one, for example, might be weighted 90% for training, 10% for validation. Point number two might be 60% training, 40% validation, and so on and so forth. It randomly weights each point, a weight for the training portion and the validation portion. Once that's done, it uses a LASSO regression technique to reduce the models and take out the terms that are not influential or not active, and comes up with the best model.

Once it's done that, it goes back, and in this case, we did that 200 times. We fit model number one as I just described, and then we repeat that process. We reassign the weightings, let the LASSO regression do its thing again, and then we end up with 200 models. When we're done, we average all those models together. That's the ensemble version, the ensemble aspect of it.

This variable weighting and ensembling technique allows the SVEM method to capture some subtleties which we might not otherwise capture in an ordinary least squares model.

Here's the plot, actual versus predicted after I ran the SVEM model. You can see that this point number 41 fits much better. Now, if I stand back and look at that, if I look at the design points, which are all the black points, it's a little noisier. It doesn't fit quite as well as it did on the 18-term regression model. But I'm a lot happier that I'm doing a better job on fitting, especially that one troublesome checkpoint.

As an aside, which I tell a lot of the researchers that I'm working with, they might look at particularly this SVEM model and say, well, all your points aren't on the line. You got a lot of scatter there. But remember, the idea behind the DOEs is not that we're going to get a high-resolution predictive tool. It's just that we want to get a predictive tool that has adequate resolution to explore the entire design space, and then we're going to leverage that to say, okay, where's the optimals? Where's the sweet spots?

Just by eyeball, that's a pretty good... When I first saw this before I did any optimization, I said, okay, I'm happy with the checkpoints. I think this model's predicting well enough on the design points that we're probably going to do a good job finding optimals. The regression fit on the left versus the SVEM fit, which is the one that we actually used.

Here is the JMP Profiler up here. It's based on the SVEM model, and right now I have it dialed into what's probably somewhere near the optimal formulation for BT production, and it's predicting a 209. Remember, our objective all we're going to do here is to maximize the millimolar titer, and at the same time minimize cost.

I'm not showing it on here, but when we ran the JMP Profiler, we also have a cost function, which is a simple deterministic function because once you know the amounts of the ingredients, then you know exactly what the cost is. If I stack those two together in the JMP Optimizer, I can say, "find me blends that maximize the production", or "Find me blends that minimize the cost", and I can also ask it balance the two.

If you look over here on the right, this is our goal grid. Down at the bottom, you have the cost, and that's on a log scale. The goal is to drive the cost as low as possible, and the goal is to drive the production as high as possible.

This lower right quadrant that I've outlined here, that would be the least desirable plot, the least desirable quadrant in that you're at higher cost, and you're at lower titers. We've identified this default formulation, and this is a formulation that was the standard formulation. It was pulled out of the literature or something that's been used before. You'll notice, at least in our grid here, it tends to be on the higher price side. That cost is in dollars per gram of BT produced.

That's our starting point. That's what we're trying to beat. We ran our JMP Profiler. We did our optimizations, and optimizations are all well and good, and we did pretty good against the checkpoints. But I'm not going to fold my hands up and say, hey, we're done at that point, because a lot of people aren't going to believe me until we go in the lab, and we take that formulation, and we make it in replicates.

That might be 5, 10, 15 replicates, depends how noisy the system is, and then we come back. The data that I'm going to show you now is not model predicted. The model predicted the formulation, but the results we're showing is what the formulation actually did in the lab, because these formulations that the model predicted did not happen to be any of the formulations that appeared in the DOE in those 55 points.

If we went for maximum production, maximum titer, it's this blue point here, and you could see it roughly doubles what the default was doing. It's still not a whole lot cheaper, but it's modestly cheaper, but it definitely makes more production, doubles it.

Then we asked the model, what's the lowest cost formulation? We came up with this one with the green dot here, and this one is down around 30 to 50 cents per gram BT, versus the default, which is up in the 300 range. That's a good two orders of magnitude reduction. If you notice the production, it's modestly better. It's not a whole lot better in the production, but it's definitely a lot lower in the cost.

These are two versions. In reality, this was a continuum. There were several points in here, and they generally performed what the model said they would do. I'm just showing you the bookends here, the maximum production and the minimum cost, both of which, in both regards, were better than the default formulation. This was considered a success, and it's led to that paper that's going to be coming out.

The bottom line: The predicted optimals were demonstrated conservatively one to two orders of magnitude reduction in the cost per gram production relative to our default formulation.

This is just one example of a DOE that's been successful. Here in 2024, my colleague presented another cell-free project where we were making a biosensor that was able to create a one-cent paper ticket. You rehydrate it, you put the bio- contaminant goes on there, it lights up. The team was able to come up with a one-cent paper ticket. I've been involved with 40 or 50 of these DOEs in the last couple of years. If you're not using them or haven't looked into them, I strongly would encourage you to do that because these are all very small experiments.

I typically work with a one to two to three, maybe a 4-day experiment, and we will usually get as much information in the one to 4 days as a research group that was operating without any structured experimentation that they might get in several months or 6 months or so. It's definitely something to look at. If it should not work, you've only lost one or 2 days.

I guess to conclude, I'll go back to my Christmas lights here. If we look at the cost and effort associated with unstructured experimentation, it's a lot heavier. It's a lot bigger than what the cost and effort associated with the DOE designs.

Yet, if we look at the information potency, the DOE designs are much more potent in the information content, much more dense in the information that they convey.

The catchphrase that I tell people is we'll get more useful information from less data. I hope that they'll challenge me on that and call me on that. That gives me an in to get with a group and help them build their first design. We've had pretty much 100% success with a group trying their first DOE. They were very happy with it.

With experimental design, collecting the right data is more important than having large volumes of disjointed data.

That brings me to my conclusion, and that's the summary. Thank you very much for listening, and I hope that any future opportunities or projects you have, you will give experimental design a try. Thank you.