topic Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations in Discussions

Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

PrakashMadhav — Sat, 10 Jun 2023 20:43:05 GMT

I am interested in creating regression equations for QSAR-related problems. The general idea is to find a linear regression equation from a pool of many (>200) independent variables or descriptors to predict one dependent variable. Typically, the number of rows/observations is small (n=50 at a minimum).

There are ways to do this using Generalized Regression in JMP. While the penalized regression methods or the simple forward or backward elimination can find a "good" linear regression equation, my concern is that these methods may not find the globally optimum solution and that they get stuck in a local minimum.

There are other techniques outside of JMP that can sample many more possible choices such as: genetic algorithms, simulated annealing, particle swarm optimization, ant/bee colony, etc. by taking subsets of descriptors, using a function to evaluate the resulting model, splitting up the subset to make and evaluate newer subsets.

I was wondering if anyone has created a way to do this within JMP or even has a way to do this outside of JMP (e.g. in R, Python, Matlab, SAS, etc.) and can perhaps think of a way to add-in this functionality within JMP? My specific need is to make linear regression equations and not neural networks, random forests, or other types of models that JMP can also make.

Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

P_Bartell — Thu, 04 Feb 2021 18:43:16 GMT

The penalized methods are one way to go. But have you considered partial least squares? You data set is wide and shallow. Tailor made for PLS. Still a linear equation. And if your predictors are correlated...even better.

Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

PrakashMadhav — Thu, 04 Feb 2021 20:55:49 GMT

An issue with PLS is that it won't select subsets of the independent variables and will just include every independent variable. This may result in overfitting issues.

Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

P_Bartell — Thu, 04 Feb 2021 21:11:47 GMT

It's not clear to me what you mean by 'subsets of variables'...if variable selection is part of your problem solving goals, PLS can be used quite nicely. You have the flexibility in JMP Pro to select the number of latent factors to work with. Then find the most influential variables within that construct...JMP Pro also has a flexible set of cross validation methods...KFold might work best for your shallow data?

Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

Byron_JMP — Mon, 08 Feb 2021 15:21:09 GMT

I know @P_Bartell loves PLS, and its a good solution in a lot of places. I'm a huge fan of the Tree Methods, and recently an add-in for XGBoost was published:

Since it sounds like you have a JMP Pro license, it might be worth your time to take a look at it.

Re: Genetic Algorithms, Simulated Annealing, etc. for Optimizing QSAR Regression Equations

Craige_Hales — Mon, 08 Feb 2021 16:19:18 GMT

None of these will help, except to suggest that JSL can be written to do simulated annealing and simulations of systems:

annealing/spring-force

simulation/visualization of a system

swarming/flocking

These are JSL intensive projects; you'll know a lot about JSL after you go down this path.