topic Re: Problem to find optimal settings for a mixture with Bayesian Optimisation in Discussions

Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Fri, 04 Sep 2026 15:25:46 GMT

With a 24 run mixture DOE I could find very good models and optimal settings for a ternary mixture and one categorical effect, this was presented in foregoing discussion; find in attachment again the analysis and below screenshots of the desirability results.

With Isomer 2 desirability result is worse; below the settings when locking Isomer 2

Starting with a 9 run candidate space filling set (see attachment Succesful 9 run) set or a 12 run Kowalski set the optimal settings with Isomer 1 can be detected.

However starting from a 6 run space filling candidate set I always get the poorer optimal settings with Isomer 2, even after several added space filling runs! What did I do wrong and how to get from this 6 run candidate the optimal Isomer 1 settings?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 10:58:22 GMT

Hi @frankderuyck,

Did you upgrade your JMP version as discussed in the previous discussion ?

Looking at your last file, when launching the BayesOpt platform from scratch only using the 6 initial space-filling runs, the models seem already quite good :

So instead of using the automatic recommendation "Replicate Best Training Run" (I deleted this run in the batch), I manually force the use of the acquisition function "Max Multimodel Std Dev" and add 1 run in the current batch, as the differentiation between Isomer 1 and 2 is difficult because of the uncertainty of the model's predictions (you can look at the confidence intervals between isomer types on the Profiler).

When relaunching the platform with this newly added run, the profiler seems to be more reliable, and the default run recommendation is done automatically with the acquisition function "Max Expected Improvement", with settings close to your optimum :

Once this automatically recommended run is added, the next option recommended by the platform is to replicate this best training run:

So given the relatively low complexity of your two responses, it is possible to start from a 6-runs space filling design and get an adequate optimum recommendation with 3 runs added, provided you think about which acquisition function is the most relevant given the learning of the models and their behaviors and you manually "enforce" this option.

Please find attached my runs situation with your 6-runs example. Done with JMP Pro 19.1.3

EDIT: I have missed the column Y in the optimization. However, I can obtain good results even when not considering it, as it seems to be negatively correlated to Column 6 2.
When considering Y and the two other Column 6 responses, you may need one extra Max Multimodel Std Dev run (so two in total, one after the other), before the Profiler shows good ordering and behavior of the different isomer types:

Once you have added 2 runs Max Multimodel Std Dev, you can start the optimization by enforcing the Max Expected Improvement criterion. You should get a solution close to the one obtained with your previous successful attemps (see file Successful 6 Run Fast Flexible Filling Design starter 2).

Hope this answer will help you,

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:03:53 GMT

Hi Victor, yes I upgraded to las JMP version. Unfortunately I can't reproduce your results; BO always starts with space filling; when replication at Isomer 2 start and I am not happy with desirability I go over to Max Multimodel St dev but I have no succes.. In an earlier webcast on BO switching to Max Max pro criterion is recommended but also this does not work. In attachment my results (there are 3 outputs Y123)

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:06:43 GMT

How do you know that in first BO run Max Multimodel St dev is necessary instead o space filling?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:09:03 GMT

Hi @frankderuyck,

I would recommend starting from scratch your scenario, leaving only the 6 space filling initial runs and deleting every other runs.
Please restart the BayesOpt platform from scratch with default settings, and force the platform to add 1 run with Max Multimodel Std Dev criterion two times. You should then have the same results for these two runs, and you can then force optimization with Max Expected Improvement, it will recommend an optimum close to the one you have with your DoE (and same inputs as the second Max Multimodel Std Dev run).

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:13:18 GMT

I can't reproduce your results when starting with Max Multimodel St Dev?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:23:32 GMT

I forgot to answer on this criterion part :

Maximize MaxPro Criterion is a model-agnostic / model-free exploration of your design space. It does not consider the outcomes and learning from the Gaussian Process model, it's only an option based on maximizing the distances between points. The goal is to add points/experiments in area of the design space where it is "empty" (maximize distance between existing points and newly recommended points).
Maximize Multimodel Std Dev is a model-based exploration of your design space. It considers the outcomes and learning from the Gaussian Process model, and recommend points in area where the prediction uncertainty is highest.

In your 6-runs scenario, the models are not bad at the beginning: R² for Y is the lowest, but is at 0,3566 so the model has been able to catch something). R² for Y2 and Y3 is very high, so model-free exploration is probably not the best next action, it is more interesting to leverage the learning of the models and try to reduce the uncertainty in their predictions.

Hope this answer will help you,

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:19:48 GMT

I'm not sure what you've done, but it seems you manage to get to the optimum, using only 1 run with Max Multimodel Std Dev criterion, and 1 run with Max Expected Improvement ? So mission successful ?

From my side, I needed 2 runs with Max Multimodel Std Dev and 1 run with Max Expected Improvement (same inputs as the second Max Multimodel Std Dev) to get close to the optimum.

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:45:15 GMT

Default R² = 0,25? At start not increase to 0,7?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:59:57 GMT

There is no hard rules on this. JMP BayesOpt development team has chosen a threshold of 0,25 to guide the model to exploration (R² < 0,25) or exploitation (R² > 0,25) phase, but this is a rule of thumb guided by their work and simulations and that shouldn't be your only motive. There is always specific scenarii where this threshold may not be adequate.

Instead, look at the model, profiler and plots, and find the best action that will improve the information gathered, either by:

Enforcing model-free exploration to ensure the design space is sufficiently explored (Max Pro criterion),
Enforcing model-based exploration to leverage the first initial models and reduce their prediction uncertainties (Multimodel Std Dev or Bayesian Desirability Std Dev criterion),
Balancing exploration and exploitation (Upper Confidence Bound criterion), or
Exploiting your models predictions and start optimizing (Max Expected Improvement or Max Bayesian Desirability criterion).

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 12:01:44 GMT

No succes after double Max Multimodel Std Dev, low desirability at Isomer 1 and BO goes back to Isomer 2

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 12:11:56 GMT

Frank, please start ONLY with your 6 initial space filling runs, without any BO scripts in the table (remove the scripts and delete any other rows).
Then, launch the BayesOpt platform with default settings, remove any automatic batch recommendation and add 1 Max Multimodel Std Dev for 2 iterations, and you should be able to get results close to your optimum with isomer 1 (and matching my testings). As soon as the two consecutive Max Multimodel Std Dev runs have been added, you should see a correct ordering of the isomer types :

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 15:32:24 GMT

Hi Victor, I followed your instructions cfr. attachment and I can't replicate your resuts? The models built with the 6 initial space filling rus are poor so I don't understand why to sart here with Max Multimodel st deviation: the outcomes are not learning a lot from poor Gaussian Process models?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 15:35:43 GMT

In your result you have only 2 outputs; there must be three

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 16:35:29 GMT

And after only 6 initial space filling runs Gaussian Y1 is very poor so additional space filling runs are required to get acceptable model performance; unfortunately, each time again, BO detects isomer 2 as optimal (?)