Bayesian optimization or similar "one experiment at a time" techniques

[#1]

I'd like to learn more about "Bayesian optimization" or similar "one experiment at a time" techniques available on JMP 18.  

Also, will Bayesian optimization be available on the non-Pro JMP platform sometime?

Thanks!

[#2 · Accepted solution]

The Bayes Op platform relies on some other features in JMP Pro for the heavy lifting. It's likely to remain a Pro feature for the future. 

Depending on what you are trying to do, a small main effects DOE with the minimal number of runs might be a useful way to start. Then, later use Augment Design to add higher order terms for active factors to fill out more of the response surface. 

[#3 · Accepted solution]

Hi @JaromW,

Welcome in the Community !

Bayesian Optimization is currently a JMP Pro platform. If you want to learn more about Bayesian Optimization in JMP Pro, I can recommend these ressources:

• Introducing the Bayesian Optimization platform in JMP Pro 
• Developer Tutorial: Bayesian Optimization 
• Bayesian Optimization: A Step-by-step JMP tutorial (Youtube video)

You can also get a quick understanding of Bayesian Optimization with this video: Basics of Bayesian Optimization (Youtube video).

To get the most of Bayesian Optimization (being able to optimize with very few iterations your product/system), a good starting dataset (with high quality information) is needed to get started. Most people start with historical data, but if you have no prior data, I would recommend starting with DoE, either using small screening designs if you have many factors (to filter out important and active factors for the BO iterations), or using small space filling designs (like Latin Hypercube) if you have few factors to get a representative and high quality starting dataset.

DoE and BO are complementary, and as pointed out by @Byron_JMP, there are also many possibilities in the DoE landscape to learn sequentially with small sized designs.

Hope this answer will help you, 

[#4]

I have two qustions/remarks with regard to bayesian optimisation:

1. Is there a way to check if the solution is not sub-optimal?

2. Is there a way to detect a robust optimal solution? 

[#5]

Hi @frankderuyck,

Here are some thoughts about your questions :

1. Not automatically. BO can get stuck in a sub-optimal area if the exploration hasn't been done completely, and depending on the difficulty to model the response and the measurement/experimental noise. I just tried an example of this situation using only the "Auto" mode to get new runs added, and very quickly the BO loop got stuck in a local optimum after a few runs, thinking it was the optimum to reach. Therefore it's a good idea to include several MaxPro criterion runs at the beginning of the BO loop to enforce exploration, before trying to reach an optimum.
   
   
2.  Same answer. Profilers like Prediction profiler and Design Space profiler, some of its options (like Sensitivity indicator), and the other modeling diagnostic tools can help you visualize and understand the possible robustness of your optimum. But if you really want to make sure your optimum is robust enough (and depending against what your optimum needs to be robust), small robustness design studies may be relevant. See this blog series about robust designs from Stat-Ease.

Hope this answer will help you,

[#6]

Again thanks Victor for clear answer! Getting stuck and being lost op a sub-optimal hill can be a problem to get lost. I understand there is a risk for ending up in a random walk requiring more runs than if one had simply started an optimal DOE? When you get stuck, is there any direction to go to the real optimum mith minimal #runs?

[#7]

Hi @frankderuyck, 

It's not necessarily a question of number of runs, but also a question of distance between the "true" global optimum you want to achieve and the one you found with BO.
In my example, I converge quicky to an optimum with an auto mode (4 runs + 1 Replicate Best Training Run added to 3 historical runs), but the optimum found in this auto mode is at 3,5 - 3,6 for the response, while the one found in a "guided" mode (enforcing 4 MaxPro runs at the beginning of the loop, for a total of 8 added runs) found the optimum at 4,5 (expected optimum). So even if the auto mode get to a solution quicker (in fewer runs), this solution doesn't seem to be the global optimum (unlike in the "guided" mode).

In my opinion, when you get stuck, the best option is to force the exploration of the experimental space, by including runs enforcing the MaxPro criterion.

[#8]

Just another point of view. If I think of the n-dimensional response surface, there are numerous local maxima, but one global optimum. The local maxima exhibit values in the response that are desired, but the local maxima is peaked (small and narrow). That is, small changes in model factors can move you from those desired values quickly. Global maxima tend to be "plateauish" (flat). This can also be referred to as robust or insensitive to small fluctuations in the model factors (I believe this is scenario II in the blog Victor referenced). In order to "get off" the local, you don't do it by level setting, you do it by finding other factors. Another dimension to "view" the surface. Factor selection moves you most effectively through n-dimensional space. So "playing around with level setting", regardless of the method, in the design space will be less effective than increasing the design space.

Additionally, my definition of robust is the absence of noise-by factor interactions. In order to develop robustness, this requires the identification and manipulation of noise at least for the duration of the experiment. To most efficiently do robust design, the noise factors must be manipulated during design factor experimentation. This is not sequential. In other words, you can't determine optimum settings for design factors, then introduce noise. It must be done simultaneously so noise-by factor interactions can be estimated. RCBD, and split-plot designs are excellent for this purpose.