Accepted Solutions
Hi @LargeDormouse25,
Concerning your questions :
- Validation portion can be helpful if you didn't specify validation (and test) sets using the Validation role. If you add a validation column with training, validation (and test) sets using the Validation role, the validation portion will be ignored :
- Validation is mostly used in this platform to enable "Early Stopping", to better assess the impact of number of trees and their depths. The validation set will be used to prevent overfitting by using too many/too deep individual trees in the forest.
Note that you can fine-tune Bootstrap Forest hyperparameters (as well as other ML models in JMP) by using a Tuning Design Table. See how to setup your table and use it for hyperparameters tuning here : Additional Example of the Bootstrap Forest Platform
If you used a tuning design table, the list of hyperparameters combinations tested and their results will be displayed in the analysis report :
Some previous conversations about Tuning design table for hyperparameters fine-tuning :
Note that Random Forest (called Bootstrap Forest in JMP) are one of the most robust supervised learning model available in Machine Learning. Unlike other possible algorithms, it's one of the few models that is less sensitive to hyperparameters tuning, and you can obtain good performances for Random Forests with the basic JMP Pro recommendation/default settings.
For more info about "tunability" of Machine Learning models (impact of hyperparameters tuning on performances), you can check this paper : [1802.09596] Tunability: Importance of Hyperparameters of Machine Learning Algorithms (arxiv.org)
Hope this answer will help you,
Hi @LargeDormouse25,
Concerning your questions :
- Validation portion can be helpful if you didn't specify validation (and test) sets using the Validation role. If you add a validation column with training, validation (and test) sets using the Validation role, the validation portion will be ignored :
- Validation is mostly used in this platform to enable "Early Stopping", to better assess the impact of number of trees and their depths. The validation set will be used to prevent overfitting by using too many/too deep individual trees in the forest.
Note that you can fine-tune Bootstrap Forest hyperparameters (as well as other ML models in JMP) by using a Tuning Design Table. See how to setup your table and use it for hyperparameters tuning here : Additional Example of the Bootstrap Forest Platform
If you used a tuning design table, the list of hyperparameters combinations tested and their results will be displayed in the analysis report :
Some previous conversations about Tuning design table for hyperparameters fine-tuning :
Note that Random Forest (called Bootstrap Forest in JMP) are one of the most robust supervised learning model available in Machine Learning. Unlike other possible algorithms, it's one of the few models that is less sensitive to hyperparameters tuning, and you can obtain good performances for Random Forests with the basic JMP Pro recommendation/default settings.
For more info about "tunability" of Machine Learning models (impact of hyperparameters tuning on performances), you can check this paper : [1802.09596] Tunability: Importance of Hyperparameters of Machine Learning Algorithms (arxiv.org)
Hope this answer will help you,
Hi Victor,
Regarding use of a tuning table for HyperParam search,
I have a mismatch between specified params
and the params reported by Model Validation.
So it looks like that "Portion Bootstrap", "Minimum Splits per Tree" and "Minimum Size Split" are not considered in the model validation. Can you confirm these param names are correct in this context ?