Bayesian optimization add-in

[#1]

• このアドインについて
• ベイズ最適化について
• アドインの使い方
• スクリーンショット
• How to use this add-in
• 変更履歴

 

このアドインについて

このアドインは、JMPのガウス過程モデルを利用して、ベイズ最適化を行うためのアドインです。

システム要件

• JMP 15以降
• 表示言語は、英語および日本語のみ

ベイズ最適化について

ベイズ最適化は、逐次的な実験を繰り返し行い、最適解を探すときに使用される手法の一つです。ベイズ最適化の主な流れは、次の通りです。

1. 適当な実験回数で、Space Filling計画（空間充填計画）（もしくは、他の計画）を立てる。
2. 実験を行う。
3. 得られたデータから、Gauss過程モデル を推定する。
4. データから推定されたGauss過程モデル g(x) （およびその共分散行列∑）より、適当な関数A(.)から計算されるA(g(x))が最大となる点 x*を求める。
5. x*を実験点として追加して、ii に戻る。

ベイズ最適化の詳細については、以下の論文等を参照してください。

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams and N. de Freitas, "Taking the Human Out of the Loop: A Review of Bayesian Optimization," in Proceedings of the IEEE, vol. 104, no. 1, pp. 148-175, Jan. 2016, doi: 10.1109/JPROC.2015.2494218.

アドインの使い方

（添付のドキュメントもご覧ください）

1. すでに行った実験データが入力されたJMPのデータテーブルを開きます。※注意：現在のアドインで対応しているのは、目的変数Yが1つの場合のみとなります。また、説明変数Xは、連続尺度のみ指定できます。
2. 「アドイン」＞「Bayesian Optimization」を選びます。
3. 列の設定、オプションの設定を行います。
4. 分析が終わると、結果のウィンドウが表示されます。候補の実験点をテーブルに追加する場合は、「テーブルに行を追加」ボタンをクリックします。
5. 次の実験を行う場合は、テーブルスクリプトの「ベイズ最適化アドインを実行」を実行すると同じ条件でベイズ最適化を行うことができます。

スクリーンショット

How to use this add-in

This add-in allows you to perform Bayesian optimization using Gaussian process model in JMP.

Bayesian optimization is one technique used when repeatedly performing sequential experiments and searching for an optimal solution.

Before conducting a real experiment, it is highly recommended to check whether it works well with some pseudo data. There are cases where it is impossible to find the optimal point well with few experiments.

1. Open the JMP data table where you entered the experimental data you have already done.
2. Select Add-Ins > Bayesian Optimization.
3. Set columns and options.
4. When the analysis is finished, a window will be displayed. If you want to add candidate experimental points to the table, click the “Add the row to table” button.
5. If you want to repeat the Bayesian optimization, click “Run Bayesian Optimization Add-in” in the table script.

For more information, please see the attached pdf file. 

変更履歴

29 September, 2023 Version 1.3 : ナゲット効果を追加するためのチェックボックスを起動ウィンドウに追加。3次構造を指定したときに、分散の計算式が正しくなるように修正。

07 October, 2023 Version 1.3.1: 小数点がカンマであるときの問題を修正

08 December, 2023 Version 1.3.2: 配合因子がある場合、エラーが表示される問題を修正

[Raaed]

Hi

Please provide me with a source for an article or book on the subject of Bayesian optimization

Regards

[yuichi_katsumur]

HI @Raaed,

Thank you for your comment. Here are some review articles I found;

S. Greenhill, S. Rana, S. Gupta, P. Vellanki and S. Venkatesh, "Bayesian Optimization for Adaptive Experimental Design: A Review," in IEEE Access, vol. 8, pp. 13937-13948, 2020, doi: 10.1109/ACCESS.2020.2966228.

B. Shahriari, K. Swersky, Z. Wang, R. P. Adams and N. de Freitas, "Taking the Human Out of the Loop: A Review of Bayesian Optimization," in Proceedings of the IEEE, vol. 104, no. 1, pp. 148-175, Jan. 2016, doi: 10.1109/JPROC.2015.2494218.

[Raaed]

Thank you very much

[summers1985]

bayesメソッドとxgboostを連携させてプラグインを作成できますか？

[yuichi_katsumur]

@summers1985 コメントありがとうございます。残念ながら、現在のバージョンでは、XGBoostのアドインと連携させる機能はありません。もし、要望が多いようでしたら、アドインを作ることを検討したいと思います。

[Peter_Hersh]

Thank you @yuichi_katsumur very useful add-in

[russ_wolfinger]

Nice work @yuichi_katsumur !    @summers1985  , although there is not currently automatic integration with the XGBoost add-in, it is straightforward to run both add-ins manually.   If you run the XGBoost Add-In with the Autotune option, it will create a table of fitting results over a fast flexible filling design.   Make sure the performance metric you want is included in the table, save it to a JMP table, then use this table as input to the Bayesian Optimizataion add-in.  This lets you profile the current results and suggest potentially improved hyperparameter settings.   Iterate between the two add-ins as needed to find ranges of optimal hyperparameters.  I tend to prefer a manual approach like this as it let's you see what is happening across the whole hyperparameter space at each step.

[Mae-Tetsu]

探索の結果から追加される実験点の範囲、条件を指定することはできませんか？

例えば、ベイズ最適化で材料A～Cの調合比率の組み合わせを最適化したい場合、A~Cの比率範囲やA~Cの合計を設定したいです。

[yuichi_katsumur]

Hello @Mae-Tetsu , Thank you for your question. Please try to set mixture column properties before performing the Bayesian optimization.

Please see the following page for detailed information for column properties.

https://www.jmp.com/support/help/ja/16.2/#page/jmp/mixture-example.shtml#ww529344 

[KOKI]

Hello,

I would like to use the Bayesian optimization add-in for the design of mixing, but is there any way to do that?

[yuichi_katsumur]

Hello @KOKI , Thank you for your question. Please try to set the mixture column properties before running the Bayesian optimization. If you have any further questions, please let me know.

[KOKI]

@yuichi_katsumur 

Thank you for your answer! I've been able to understand your suggestion by actually running Bayesian add-in with a sample data set. The issue has been resolved.

[ThomasBaah]

How do I know the Rsquare after finding candidate experiment points with Bayesian Optimization add-in? 

[yuichi_katsumur]

Hello @ThomasBaah ,

Thank you for your question. This add-in uses the Gaussian process in JMP. One measure of goodness of fit is how well the points lie along the 45-degree diagonal line in the actual versus predicted plot. JMP does not report the R2 statistic for the Gaussian process. For information on the Actual vs. Predicted plot, see the following page.

https://www.jmp.com/support/help/en/18.0/#page/jmp/actual-by-predicted-plot-2.shtml# 

[ThomasBaah]

@yuichi_katsumur Thank you!

Is there any better way to extract plots in bayesian optimization add-in? I'm bit new to JMP

[yuichi_katsumur]

@ThomasBaah  You can use either of the following methods.

https://www.jmp.com/support/help/en/18.0/#page/jmp/copy-and-paste-reports.shtml# 

https://www.jmp.com/support/help/en/18.0/#page/jmp/save-reports-as-powerpoint-presentations.shtml# 

[ThomasBaah]

This is the actual plot I'm getting from the Gaussian model. Though the predictions looks reasonably accurate, how do I validate this? Also, I would like to know how this actual plot was generated: does it use the training data or the testing data? I understand that in typical ML, the dataset is normally split into training and testing data. I would like to know more about this, please.

Thanks.

@yuichi_katsumur 

[yuichi_katsumur]

Hi @ThomasBaah ,

Regarding how the plot was generated, the actual plot uses jackknife predicted values. However, these values are not true jackknife values because the model is not re-fitted for each excluded Y-value. Instead, A jackknife predicted value is obtained from a model that doesn't include that row in the table. This graph is good for assessing model fit; the closer the points form a line, the better the model. I'm sorry, but I cannot comment on how well they should line up in a straight line.

For your information, the true jackknife approach is used to exclude rows from the calibration dataset, refit the model, and validate the model to predict the excluded observations (again, not like the Gaussian process in JMP).

[ThomasBaah]

@yuichi_katsumur Thank you.

I would also like to know if the x,factor has column limit. I have 6 columns for x,factor and after running, there's no predicted targeted value in the prediction profiler. Sometimes there's a note saying column marked * are not used it the GP model...
is it possible to have all my x,factor variables on the prediction profiler? 

[yuichi_katsumur]

@ThomasBaah ,

One possible approach is to set a minimum theta value in the dialog.

See the following page for details on the minimum theta value.

https://www.jmp.com/support/help/en/18.0/#page/jmp/launch-the-gaussian-process-platform.shtml# 

[ThomasBaah]

@yuichi_katsumur Thanks , that was helpful.
How does one specify the search space when using Bayesian Optimization to find candidates for the next experiment? For instance, I want the optimized value of material A to be within 500-700.

[yuichi_katsumur]

@ThomasBaah  Thank you for your question. 

You can specify a factor range constraint by creating a table script named "Constraint" on the data table in advance.

For more information, see the following page;

https://www.jmp.com/support/help/en/18.0/#page/jmp/linear-constraints.shtml#ww343016