Demystifying the Bayesian Optimization Platform in JMP Pro (2026-US-30MP-2942)

Welcome. Thank you for coming to this talk. My name is Kasia Dobrzycka. I'm a research statistician and developer on the JMP modeling team. I'll also give a shout-out to Bradley Bowen from Global Education, who did an incredible job producing a lot of the visuals you'll see today.

To start, the goal of this talk is to provide some clarity on common questions and ambiguity surrounding the Bayes Opt platform. For that purpose, I'll do a quick refresher on the fundamentals of the platform and a simple auto mode demo. Then I'll go through a sorted FAQ topics we've heard about related to batch selection, acquisition functions, and other topics. Finally, we'll go a bit more in-depth into how and when to use custom datasets to leverage Bayes Opt's flexibility for solving complex problems.

To set a baseline for what Bayes Opt is and does, the point of Bayes Opt is to select new points to sample iteratively throughout our experiment. The process can be summarized by these five steps. We collect our data, we fit a surrogate model, optimize the acquisition functions to choose new sample points, run the experiment with the newly proposed settings, and repeat until we're satisfied.

The two major mathematical components involved in this workflow are the surrogate model and the acquisition function. A surrogate model is a prediction model that is meant to simulate our system. The idea is we can sample predictions from this model a lot faster and a lot cheaper than we can sample our actual system. In computer experiments, this model is typically a Gaussian process model, which is also the case for us.

A Gaussian process or GP is a distribution over functions where any finite collection of function values follows a multivariate normal distribution. That took me a solid year to wrap my head around in grad school, so don't worry if it sounds like math gibberish. You can just think of a Gaussian process as a function that connects the dots between observations and additionally quantifies the uncertainty at each input value.

In this simple graph of a Gaussian process with one input and one output, we call the line that connects the dots the posterior mean function. It's what we use for prediction. These blue lines represent the uncertainty of our predictions at each point. The individual blue lines essentially represent a possible alternative line that could connect the dots instead.

The big takeaway regarding GP behavior is that uncertainty is near zero at the red points, which are our existing observations, while the uncertainty gets bigger as we go further away from them. It's a distance-based approach where the uncertainty in predictions grows as you get further away from existing observations. The other major tool is the acquisition function. An acquisition function is any function we use to select our next point. This is done by defining some function that quantifies what we want to get out of our next sample and optimizing it based on our surrogate model predictions.

It's either a function of the inputs or a function of our prediction function, which itself is a function of the inputs and observed outputs. We'll get into the details of this component later. Using these two tools, Gaussian processes and acquisition functions, we have a workflow that can handle very complex problems. We can deal with multiple responses with competing goals by defining a posterior desirability function that's a function of multiple models instead of just the one. We can handle any mix of factors and different types of constraints. By using a conditional posterior, we can select multiple batch runs per iteration instead of traditional one-at-a-time sampling.

The reason we can handle these complex situations that more rigid, traditional models have trouble with is because of the limited statistical machinery involved in Gaussian process models. However, we pay for this convenience with the loss of interpretability. While a slope of a simple linear regression directly describes the relationship between input and output, the hyperparameters of Gaussian processes do not provide such convenient inferential information.

Let's breeze through an example of a simple optimization problem solved using the Bayes Opt platform. We have a simple problem with three inputs: glucose, oil, and nitrogen, and a single response, yield. We start with a simple starting sample of three points selected at random. We run the Bayes Opt platform, go directly to the suggested next run and export to the table. We can see here Bayes Opt decided to use the MaxPro space-filling criterion as the acquisition function for this iteration, likely because a model fit to three points is likely less than stellar. You can see how far away this fourth point was in the plot here. We can also see that the response is actually much better than the starting data, so we're already on the right track.

But we definitely want to keep sampling to try to optimize further. For the next iteration, Bayes Opt chose to use maximum expected improvement as the acquisition function, indicating our fit has significantly improved. We sample at that point, and we can see that compared to the previous point, oil and nitrogen stayed about the same while glucose changed a lot, and we still maintain a relatively high yield. Jumping ahead a few more iterations, we see Bayes Opt continued to choose expected improvement until the sixth iteration, where the platform is confident enough in the optimum we've already found that it suggests we replicate that point to confirm our optimum. We do so just in case we have a very noisy system to make sure that the previous observation wasn't just a fluke.

That simple example leaves us with a lot of questions, right? How are we choosing these acquisition functions? What do they mean? Why do we not use a larger batch size, et cetera? Let's jump into some of those topics. Firstly, how do we use acquisition functions? Again, the point of the acquisition function in general is to define some property of the model that we want our next selected point to optimize.

If you've heard the concept of objective functions, those are quite similar. An acquisition function is a function that quantifies the objective of our next point. But that's still quite abstract, right? To get more specific, acquisition functions generally either focus on exploitation, exploration, or a built-in balance between the two. Exploit and explore are linguistic terms you may not have heard in this context before. All we mean is that we're either marching towards our goal by exploiting the information we already have to improve our optimum, or we are exploring at areas we have less information about to make sure we haven't overlooked potential unobserved optima.

Looking at this simple Gaussian process model here, an acquisition function prioritizing exploration might choose a point that has very wide prediction intervals, like in these areas, with the goal of reducing model uncertainty, while an exploitation-focused acquisition function would tell us to search for an optimum around this peak. To get even more specific, let's look at a couple of examples. One popular acquisition function is the upper confidence bound or UCB. It's the upper bound of the confidence interval for the desirability of the prediction at a given factor setting x.

You've got your mean predicted value and your weighted prediction standard deviation. The weight in this context controls the trade-off between exploration and exploitation. If the mean is very high, UCB will choose to exploit that area further. If the variance at a certain point is very high, UCB might choose to explore that area further to try to reduce that prediction uncertainty.

In this example graph we've seen, UCB would choose this high-mean, medium-uncertainty point rather than these other example points that have, for example, a high mean but low uncertainty, or a low mean and very high uncertainty. However, you can imagine if our variance was even larger, that low-mean, high-uncertainty point could become preferred instead.

Another acquisition function is Bayesian desirability standard deviation. Much simpler, this is just choosing the point with the highest prediction standard deviation. In this example, using this acquisition function would instead yield this point with the highest prediction variance. If you have multiple points of equal high uncertainty, like this other point towards the right of the graph that might be about the same, you would just choose between them at random. For those of you that prefer thinking about things in more mathematical terms, you can think of the Gaussian process model for a single response as a function of inputs X given the observed data.

The desirability is a function of predictions. For example, if we are minimizing y, g would be increasing as y is decreasing. An acquisition function is then a function of the desirability. This is a bit of an oversimplification, but hopefully it helps you understand the relationship of all these terms and functions we're throwing around. Right now in Bayes Opt, we have six acquisition functions available built into the platform. Of these, maximizing Bayesian desirability would be considered to be exclusively exploiting. We're picking the point with the largest mean predicted desirability.

The next three are focused directly on exploration. Like we discussed, maximizing Bayesian desirability standard deviation chooses the point with the greatest prediction uncertainty, so we're purely exploring points we're really unsure about. Multimodal standard deviation maximizes the overall model standard deviation across all the models. MaxPro is a bit different because instead of looking at prediction uncertainty, it just looks at the x values we've collected so far and tries to fill out the biggest gaps left over in the design space. If you've heard of minimax or maximin designs, it's conceptually similar to those.

EI and UCB at the end here are a bit more complex. They are popular in computer experiments because they have built-in machinery to choose whether to lean more into exploration or exploitation all by themselves based on the model. As we saw with UCB, the weight parameter basically determines how much we value exploration over exploitation. Generally, you have one option that explicitly exploits or optimizes, three that focus on exploring the design space further, and two that choose a mathematical balance between exploring and exploiting based on that model.

Those are nice generalizations, but when do we actually use each one specifically? If you trust your model and you just want to optimize, you would maximize Bayesian desirability directly. When we want to reduce uncertainty in the overall desirability, we would use the standard deviation. When an individual response model is weak or we just want to improve models in general, we might use multimodal standard deviation. When we really don't trust our model at all and we don't have enough information to really go off of, which is indicated when R-squareds are bad, we would use the MaxPro criterion.

If you want to begin refining our search without completely overlooking areas that we don't have information about, we would use expected improvement. For a slightly more aggressive alternative, we could use UCB. In my experience, EI and UCB tend to generally agree in their decision-making. UCB is much simpler to optimize, but people really like the label of expected improvement. Do with that information what you will. I tend to be a little bit biased.

Now that we understand acquisition functions a little better, let's talk about how JMP actually uses them. Auto mode has three different phases. You can think of it as choosing gears when driving a car. When we're just starting up and our models haven't gotten going yet, Bayes Opt will choose to purely focus on exploration with the goal of improving R-squareds. Once that goal is satisfied, we can jump into second gear: refining the model. We can start using our model for decision-making to choose our next point, but we feel that there's still room for improvement in either the surrogate model or the optimum so far.

Once we're close to our destination, our models are up and running, and we've gotten closer to the optimum, we enter the final phase of confirming and challenging our optimum. We want to make sure previous observations weren't outliers or a fluke.

For each phase, we use different acquisition functions to target those goals. In the space-filling mode, our models are so weak that even prediction variances aren't really enough to differentiate between factor settings, and we don't really trust those measurements anyway. Therefore, we stick with using the MaxPro criterion for that phase. In the second phase, we use a mix of acquisition functions. The first point in the batch uses expected improvement directly. When you see one-at-a-time sequential experimentation, you'll often see EI being used to pick that single point at each step.

In our case, if we can afford to run a bigger batch, we then spend the next two points in the batch exploring the areas we're most uncertain about. Any remaining points past that are used to improve overall variance to continue improving our model. Now, maybe you're wondering, "Kasia, why don't we just use EI for all the runs? We love EI. EI is great." The reason Bayes Opt can choose multiple points at once is because the acquisition functions are functions of the posterior or conditional desirability of prediction.

In other words, when we're choosing our second batch point, we are conditioning our predictions, and more importantly, our prediction uncertainty estimates, on the assumption that we will have already collected our first batch point. That means we can update our uncertainty values, but not our mean predictions, because we haven't actually collected the response at that batch point.

For EI, and similarly for UCB, that means we're only updating one component of the acquisition function, and it'll likely continue selecting similar points. However, for the exploration-focused acquisition functions, those are almost entirely dependent on X rather than Y, so the conditional posterior is a bit more informative for them. Finally, in confirm and challenge mode, for the first point, we're directly optimizing the Bayes desirability and using any other available batch points to continue exploring areas with the highest uncertainty, since those are the areas most likely to be hiding a nugget of gold we may have overlooked.

In this mode, you'll likely eventually see "Replicate of best point" showing up. This means that the point with the highest desirability is not statistically significantly different from the already observed optimum, so we're pretty much done searching. Now we know how auto mode selects batches, but we know that the available batch size does impact how points are selected. Five batches of two points each will not be selected the same way as doing two batches of five points each.

Changing the size is fairly simple in Bayes Opt. Bayes Opt runs auto mode when you launch the platform, so you can go straight to your automatically generated batch points and go right back to experimenting. In that scenario, you just have to enter your desired batch size into the launch dialog while you're loading your input and output columns.

In 19 versions of JMP Pro, you can use the Auto Select Rows button under the candidate set outline to rerun auto mode with changed batch size after launch. In 20, we're adding a quality-of-life change to integrate this pop-up UI directly into the batch table outline, so that will hopefully make it easier and simpler to play around with auto mode instead in the future. That's how to actually change it, but you still need to decide what size batch you want.

In general, you can think of batch size as a tuning parameter of the Bayes Opt process. As batch size increases, cost increases, but exploration also increases. Also, note that the batch size does not have to stay the same throughout our experiment. Early on, we know one or two points may not be enough to substantially improve our models.

That's fine. We can run larger batches using the space-filling mode until we're out of that phase. Then we can reduce our size to focus on optimization instead of wasting, for lack of a better word, resources on exploration, even though we're already satisfied with our models. At the end of the day, the minimum batch size is one, and the maximum is whatever the all-at-once sample you would have done instead of this. There's no optimal batch size we have a convenient closed-form solution for. It's purely a decision made based on the context of your resource availability and preferences. Just know that there's no wrong answer.

Choosing batch size of one may end up taking you more iterations because you've done the minimum amount of exploration and happened to get unlucky, while choosing a large batch size may end up taking more total runs because you spent a lot of points exploring areas that didn't end up being useful. At the end of the day, here are some reasons you may have for leaning one way or the other on batch size. In the case where you have a machine or process that collects n samples at a time, you might want a batch size of exactly size n.

In other cases, you might start a project with a large amount of starting data or historical data. In that case, you might want a smaller batch size because you don't have as much exploring left to do. You've already mapped out the design space pretty thoroughly. On the opposite end of the spectrum, if you have little to no starting data, you might prefer a larger batch size to get all that exploring out of the way at once. In this scenario where a new ingredient just became available and you have existing data, but none of it includes this new ingredient or this new variable altogether, you're going to have to explore that dimension or new factor setting, so you might want a larger batch size at that iteration.

If you need to perform follow-up analysis on the data and you don't mind exploring areas away from the optimum, you might prefer a larger batch size. If individual runs are extremely time-consuming, but you can perform runs concurrently, you might want a larger batch size. While on the opposite end of the spectrum, again, if individual runs are extremely expensive, you might want to reach the optimum as cheaply as possible, then a smaller batch size might suit your goal better.

The takeaway here is that batch size is a purely practical decision and not a statistically driven one. Don't be afraid to experiment. We've covered auto mode and batch size, but what are other ways you can select batches beyond blindly trusting auto mode? Like we said, for simple problems, we can rely on that auto mode and breeze through our optimization problems. If we want to dig into the response models, we can look at the profilers in the Batch Selection tab to see the overall trends of acquisition functions and predictions across different input values and choose our batch points manually that way.

For the most complicated problems, we can use custom candidate sets to choose points. Basically, using a candidate set is like doing a brute-force search instead of directly optimizing through the profiler. Obviously, that's not an ideal way to optimize, but it allows us to leverage computing power to compensate for complex limitations. Let's get into the nitty-gritty you came to see about custom candidate sets. First of all, when and why would you choose to go through the trouble of manually setting up a custom candidate set when we have the lovely Bayes Opt platform to automatically generate one for us?

Well, like we mentioned earlier, there are situations that are too complicated to handle automatically. These situations largely fall into one of two categories. First off, you have complex constraints. When constraints get involved, we rely on the profiler's built-in random sample generator. If the profiler can't handle your constraint, you will have to handle it yourself. The other category is situations the platform can't intuit from your data alone. If you have a new ingredient you want to use or a previously existing ingredient is no longer available, the platform can't read your mind on that.

If you have subject matter knowledge and you know specific combinations of settings or certain ranges aren't going to be optimal, even if they're feasible, the platform can't intuit that from your data table. If you have hard-to-change factors, the platform should be able to read your column properties, but for the time being, it cannot, unfortunately, so those will have to be set up manually. To do so, you can take the auto-generated candidate set, export it to the data table, and change all those factor levels manually.

Don't be afraid. There's no sacred mathematical perfection baked into the auto-generated candidate set. It is, like I said, a glorified grid over the design space. It is a brute-force, last resort for when your requirements can't be handled directly by the profiler. Don't be afraid to go full mad scientist and mold and sculpt it into the shape you need it to cover for your problem.

With the time remaining, let's take a look at an example of how we can tackle a more messy problem with complex mixture constraints by using custom candidate sets. If you saw our talk last year about mixture constraints and some new JSL functions available in JMP, this might look a little familiar to you. What if we have a polymer formulation problem where we have a recipe with multiple submixtures? The result of this formulation process needs to aim for an inherent viscosity of four, our particular target, while we have to maximize these other three metrics of the resulting polymer.

We have three submixtures of types of ingredients. In the case of acids, we have five options, and we can only choose 1–2 components of each one. In the case of glycols, we have six options, and we can choose one through three of those components. Other variables include acid number, hydroxyl number, and type of reactor, which is a process variable.

Here's just a quick overview of the actual constraints and bounds for each of these. Basically, each of the categories' proportions need to add up to one, and each individual component needs to be less than one. This is true across both of the mixture categories. We can use this new random linearly constrained uniform JSL function to generate our candidate set. It will be available in version 20, and it's specifically built to handle mixture of mixtures and cardinality constraints. Assuming we've done that, we can jump into solving our problem.

We can launch JMP Bayesian Optimization with our previously collected data for our starting set. We fill in our factors and responses and input our desired batch size. In this case, we have a machine that processes two batches at a time, so we're going to do that as our batch size. Once we get the platform open, we load in our candidate set from data table. With the auto mode, when you load in a candidate set, Bayes Opt automatically selects a new batch of the size that we requested on launch.

When we export our new batch, we want to include the original candidate set row indices for our reference, so we know which points are being selected from that candidate set, since we are going to be reusing that candidate set at each iteration. Now, like with the simpler example previously, we can see both the new batch points and the documentation columns added to the data table. We can then collect our new observations in the new batch and use the startup script for next batch to relaunch. In 20, this will automatically save our loaded candidate set, so you won't have to load it in manually. But until then, unfortunately, in 19, we will have to manually load the candidate set again.

From there, we can just repeat the process until we're satisfied. In this case, the problem gets a little more complicated. We don't have a clear stopping point. It's a complicated problem with several responses. After 24 additional runs, we see a solution. When we see "Replicate best training run," that's when the platform's indicating that it can't find a better option to try. However, using the Model Summary tab of the Bayes Opt platform, we can evaluate our results so far.

In this case, the point we're actually replicating has, although the same factor settings, wildly different observed desirability. That means that our optimum in specific might be highly noisy or the responses are highly variable. Let's say continue with our batches to see if we continue replicating the same run to confirm our results. We can see, once again, we start replicating, but we're replicating a different point each time.

In this case, we start seeing a different point performing more consistently. Looking at the Desirability Run Chart, we can see that both of these replicated points score higher on the desirability. They're not the highest observed desirability we've seen so far, but they are staying consistently high even after replicating. We can see that the auto mode in Bayes Opt continues selecting the same point as the optimum in this case, and so we can be more confident that we're done searching.

With that example out of the way, here are just a few more general tips for some more common questions that didn't really fit into any of the broad categories. I've gotten asked a few times about general recommendations for number of starting runs, not just batch size. Generally, we recommend a minimum of p factor runs, or the number of factors, at least, and up to two times the number of factors. Generally, similarly to the batch size question, it's a matter of how much time are you willing to spend exploring through Bayes Opt iterations versus how much data can you afford to start with?

Like we said, when should you stop using Bayes Opt? That's really a qualitative decision over a quantitative one. It's up to you to define your stopping condition. You generally want to stop when your product or development goals are satisfied, and you can use the "Replicate best training run" reason as an indicator that you can't improve significantly more. What happens if I don't specify factor ranges in the column properties? Bayes Opt will default to the observed range in the data table. Like we discussed, batch size is changeable in the middle of Bayes Opt. You don't have to use the same batch size at every iteration throughout your experiment.

Finally, a common question is using the custom candidate set like we discussed earlier, and yes, we can always change our candidate set. There are no rigid, fixed qualities that we require from our candidate set. That's all I have. Thank you so much for attending. I hope you enjoyed.