topic Re: Problem to find optimal settings for a mixture with Bayesian Optimisation in Discussions

Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Fri, 04 Sep 2026 15:25:46 GMT

With a 24 run mixture DOE I could find very good models and optimal settings for a ternary mixture and one categorical effect, this was presented in foregoing discussion; find in attachment again the analysis and below screenshots of the desirability results.

With Isomer 2 desirability result is worse; below the settings when locking Isomer 2

Starting with a 9 run candidate space filling set (see attachment Succesful 9 run) set or a 12 run Kowalski set the optimal settings with Isomer 1 can be detected.

However starting from a 6 run space filling candidate set I always get the poorer optimal settings with Isomer 2, even after several added space filling runs! What did I do wrong and how to get from this 6 run candidate the optimal Isomer 1 settings?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 10:58:22 GMT

Hi @frankderuyck,

Did you upgrade your JMP version as discussed in the previous discussion ?

Looking at your last file, when launching the BayesOpt platform from scratch only using the 6 initial space-filling runs, the models seem already quite good :

So instead of using the automatic recommendation "Replicate Best Training Run" (I deleted this run in the batch), I manually force the use of the acquisition function "Max Multimodel Std Dev" and add 1 run in the current batch, as the differentiation between Isomer 1 and 2 is difficult because of the uncertainty of the model's predictions (you can look at the confidence intervals between isomer types on the Profiler).

When relaunching the platform with this newly added run, the profiler seems to be more reliable, and the default run recommendation is done automatically with the acquisition function "Max Expected Improvement", with settings close to your optimum :

Once this automatically recommended run is added, the next option recommended by the platform is to replicate this best training run:

So given the relatively low complexity of your two responses, it is possible to start from a 6-runs space filling design and get an adequate optimum recommendation with 3 runs added, provided you think about which acquisition function is the most relevant given the learning of the models and their behaviors and you manually "enforce" this option.

Please find attached my runs situation with your 6-runs example. Done with JMP Pro 19.1.3

EDIT: I have missed the column Y in the optimization. However, I can obtain good results even when not considering it, as it seems to be negatively correlated to Column 6 2.
When considering Y and the two other Column 6 responses, you may need one extra Max Multimodel Std Dev run (so two in total, one after the other), before the Profiler shows good ordering and behavior of the different isomer types:

Once you have added 2 runs Max Multimodel Std Dev, you can start the optimization by enforcing the Max Expected Improvement criterion. You should get a solution close to the one obtained with your previous successful attemps (see file Successful 6 Run Fast Flexible Filling Design starter 2).

Hope this answer will help you,

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:03:53 GMT

Hi Victor, yes I upgraded to las JMP version. Unfortunately I can't reproduce your results; BO always starts with space filling; when replication at Isomer 2 start and I am not happy with desirability I go over to Max Multimodel St dev but I have no succes.. In an earlier webcast on BO switching to Max Max pro criterion is recommended but also this does not work. In attachment my results (there are 3 outputs Y123)

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:06:43 GMT

How do you know that in first BO run Max Multimodel St dev is necessary instead o space filling?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:09:03 GMT

Hi @frankderuyck,

I would recommend starting from scratch your scenario, leaving only the 6 space filling initial runs and deleting every other runs.
Please restart the BayesOpt platform from scratch with default settings, and force the platform to add 1 run with Max Multimodel Std Dev criterion two times. You should then have the same results for these two runs, and you can then force optimization with Max Expected Improvement, it will recommend an optimum close to the one you have with your DoE (and same inputs as the second Max Multimodel Std Dev run).

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:13:18 GMT

I can't reproduce your results when starting with Max Multimodel St Dev?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:23:32 GMT

I forgot to answer on this criterion part :

Maximize MaxPro Criterion is a model-agnostic / model-free exploration of your design space. It does not consider the outcomes and learning from the Gaussian Process model, it's only an option based on maximizing the distances between points. The goal is to add points/experiments in area of the design space where it is "empty" (maximize distance between existing points and newly recommended points).
Maximize Multimodel Std Dev is a model-based exploration of your design space. It considers the outcomes and learning from the Gaussian Process model, and recommend points in area where the prediction uncertainty is highest.

In your 6-runs scenario, the models are not bad at the beginning: R² for Y is the lowest, but is at 0,3566 so the model has been able to catch something). R² for Y2 and Y3 is very high, so model-free exploration is probably not the best next action, it is more interesting to leverage the learning of the models and try to reduce the uncertainty in their predictions.

Hope this answer will help you,

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:19:48 GMT

I'm not sure what you've done, but it seems you manage to get to the optimum, using only 1 run with Max Multimodel Std Dev criterion, and 1 run with Max Expected Improvement ? So mission successful ?

From my side, I needed 2 runs with Max Multimodel Std Dev and 1 run with Max Expected Improvement (same inputs as the second Max Multimodel Std Dev) to get close to the optimum.

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 11:45:15 GMT

Default R² = 0,25? At start not increase to 0,7?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 11:59:57 GMT

There is no hard rules on this. JMP BayesOpt development team has chosen a threshold of 0,25 to guide the model to exploration (R² < 0,25) or exploitation (R² > 0,25) phase, but this is a rule of thumb guided by their work and simulations and that shouldn't be your only motive. There is always specific scenarii where this threshold may not be adequate.

Instead, look at the model, profiler and plots, and find the best action that will improve the information gathered, either by:

Enforcing model-free exploration to ensure the design space is sufficiently explored (Max Pro criterion),
Enforcing model-based exploration to leverage the first initial models and reduce their prediction uncertainties (Multimodel Std Dev or Bayesian Desirability Std Dev criterion),
Balancing exploration and exploitation (Upper Confidence Bound criterion), or
Exploiting your models predictions and start optimizing (Max Expected Improvement or Max Bayesian Desirability criterion).

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 12:01:44 GMT

No succes after double Max Multimodel Std Dev, low desirability at Isomer 1 and BO goes back to Isomer 2

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Mon, 07 Sep 2026 12:11:56 GMT

Frank, please start ONLY with your 6 initial space filling runs, without any BO scripts in the table (remove the scripts and delete any other rows).
Then, launch the BayesOpt platform with default settings, remove any automatic batch recommendation and add 1 Max Multimodel Std Dev for 2 iterations, and you should be able to get results close to your optimum with isomer 1 (and matching my testings). As soon as the two consecutive Max Multimodel Std Dev runs have been added, you should see a correct ordering of the isomer types :

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 15:32:24 GMT

Hi Victor, I followed your instructions cfr. attachment and I can't replicate your resuts? The models built with the 6 initial space filling rus are poor so I don't understand why to sart here with Max Multimodel st deviation: the outcomes are not learning a lot from poor Gaussian Process models?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 15:35:43 GMT

In your result you have only 2 outputs; there must be three

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Mon, 07 Sep 2026 16:35:29 GMT

And after only 6 initial space filling runs Gaussian Y1 is very poor so additional space filling runs are required to get acceptable model performance; unfortunately, each time again, BO detects isomer 2 as optimal (?)

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Tue, 08 Sep 2026 06:44:59 GMT

I have the same problem starting from initial 10 run space filliing DOE cfr. attachment, what went wrong? We checked in the lab and the correct optimal solution is with Isomer 1. Isomer 2 gives worse results and Isomer 3 is totally unacceptable. Impprtant to note is that starting from 9 run initial space filling DOE's BO quickly converts to the right Isomer 1 setting!

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Tue, 08 Sep 2026 06:46:03 GMT

Succesful 9 run space filling DOE. Strange that with lower #runs results are OK

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

Victor_G — Tue, 08 Sep 2026 09:42:36 GMT

Some answers to your many replies:

You're able to replicate the results, see the input and responses values for the second Max Multimodel Std Dev run in your file and mine. If after this second run you force the BayesOpt platform to add a run with Max Expected Improvement (instead of the automatic recommendation of MaxPro space filling run criterion), this second added run will be repeated, and is close to the optimum you're expecting with isomer 1.
I had three outputs in the files I send: Y, Column 6 1 and Column 6 2.
MaxPro Space filling criterion run are indirectly improving the model by forcing the exploration of the design space by maximizing the distances between existing design points and new ones. The effect on the model is not certain, depending on the complexity of the response. This is why in the JMP Help and in my answer I mention this option as a "model-free" exploration option. On the other side, Multimodel Std Dev and Bayesian Desirability Std Dev are model-based exploration of the design space : they use the model and its prediction uncertainties to recommend experiments in area where uncertainty is the highest.
So to get acceptable model performance, you need both :
- Space filling runs to ensure a good coverage of the design space and make sure the model is able to generalize the learned behavior, and that this behavior is representative of the real phenomenon in the design space.
- Bayesian/Multimodel Std Dev runs to ensure the reduction of prediction uncertainties to reach an acceptable level of precision.
  
  Please see my previous explanation here.

It's requiring a lot of time to look at all your different scenarii. What you have to remember is :

Bayesian Optimization has the objective to reach your targets: as soon as an acceptable result is reached (based on your conditions), the loop will converge to it, no matter how global or local the optimum may be. So depending on how "tight" your target specifications are (and how complex the response surface is), you may have a premature convergence or not.
By default in the BayesOpt platform, the exploration/exploitation part is automatically set depending on the R² threshold (default = 0,25): if the model R² is below the threshold, the recommendation will be MaxPro space filling runs. If it is above the threshold, it will start the exploitation with Max Expected Improvement criterion runs. The other options need to be enforced manually.
I would highly recommend spending time looking and investigating your model; BayesOpt may look like an automatic experimentation loop, with good default experiments recommendation, but you can have better results by enforcing specific runs needed for the models.
Your different scenarii with different number of space filling runs provide a different amount and precision of information. The less runs you have when starting, the harder the task, and so the more information and care your should bring to the models, by providing adequate runs with the right information, either by exploring the design space with MaxPro space filling runs or improving model uncertainties with Bayesian/Multimodel Std Dev runs. The strategy and choice of acquisition functions, number of parallel or sequential runs, etc... should be guided by the information you have, the precision and learning of the models obtained.

Hope this answer will help you understand your different results. Please read carefully the files I have sent earlier, you were able to reproduce my results, you just need to enforce the same runs using the Profiler and dedicated acquisition functions (and not let the automatic selection works that give you a MaxPro space filling last run).

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Tue, 08 Sep 2026 09:48:04 GMT

Hi Victor in your succeful 6 run initial abov I only see 2 outputs Column 6.1 and Column 6.2 see below. There i a 3rd one Y, I don't see it

Incuding Y I can't detect the right optimal settings also not with 10 run see attachment (I recoded the 3 outputs as Y1, Y2 and Y3)

What did I do wrong? It is important for us to understand.

With 9 intial runs it always works perfect, why?

Re: Problem to find optimal settings for a mixture with Bayesian Optimisation

frankderuyck — Tue, 08 Sep 2026 10:34:05 GMT

"Space filling runs to ensure a good coverage of the design space --> I have used fast flexible space filling, OK? Is assessment of the initial starting DOE is necessary? I saw a number of webinars where one starts from just a vey limited #initial runs, sometimes even from an OFAT; guess this will only work when model is simple and when data are not too noisy, correct? This is not always the case and we don't know in advance.

Make sure the model is able to generalize the learned behavior, and that this behavior is representative of the real phenomenon in the design space" --> In my examples models have R² > 0,8, are there other criteria?