Hi @frankderuyck,
Great to know !
I use Neural Networks as the last modeling option possible, there are many classical models and ML algorithms more simple that perform equally well (and sometimes better). These other options are also faster to compute, don't require extended knowledge, tests and validation to finetune them (unlike NNs where you have to specify the architecture, with the number of layers, type of functions, boosting or not, ... or using the Neural Network Tuning AddIn) and are easier to interpret (in terms of response profiles and relative factors importance).
On tabular data, if the only model able to perform well is Neural Networks, there are good chances that you might have other problems, in terms of data quality, representativeness, missing values/outliers, repeatability/reproducibility, etc...
Tree-based models provide a good performances benchmark and are often a safe and reliable option, see the article "Why do tree-based models still outperform deep learning on tabular data?" (2022) https://arxiv.org/abs/2207.08815
In your screenshot, it seems the optimal SVM model found with the Tuning design is quite interesting :
- It uses 10 support vectors, so less than half of the points are used to build the "boundaries"/vectors used in SVM model, which may be a good sign of low complexity and good generalization properties,
- The curvature value (gamma) is quite low, so the response profile and boundaries may not be over-complex and highly curved (so again, low complexity and good generalization properties),
- The penalty (cost value) seems quite high, but with a low-size dataset this is often the case in practice, as any misclassified/mispredicted point will greatly increase the misclassification rate/RMSE : 1 misclassified point out of 25 = 4% misclassification rate, compared to higher dataset size where one misclassified point would represent less than 1% misclassification rate. In some situations, it can be interesting to choose a model with a slightly larger error (RMSE/Misclassification rate), but with a lot less support vectors : you degrade your predictive performances on the training set in favor of a less complex model that could be able to generalize better on new data.
You can use the table generated by the Tuning design, with the performance metrics, to better assess the relative importance and contribution of the two hyperparameters on the performance metrics (Misclassification rate for classification, or RASE and R² for regression).
As always, it is recommended to validate your model with new validation points, to make sure your model doesn't overfit.
Victor GUILLER
L'Oréal Data & Analytics
"It is not unusual for a well-designed experiment to analyze itself" (Box, Hunter and Hunter)