topic Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA in Discussions

Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

Nazarkovsky — Sat, 10 Jun 2023 20:45:56 GMT

Dear colleagues,

I have encountered a situation to deal with PCR (Principal Components Regression) and Partial Least Square Regression (PLSR). Unfortunately, no info about PCR and PCA-based PLSR I did not find in the tutorials.

I feel there must be a manual simple way to arrange predictive modeling using the results obtained after the PCA option in JMP Pro.

I would kindly appreciate the advice how to operate the data of PCA using predictive modeling in PCR & PLSR.

Many thanks!
Michael

Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

Mark_Bailey — Tue, 13 Jul 2021 13:25:44 GMT

You might start with the JMP Help about PCA, which talks about the JMP tool.

This book might be helpful as it goes into methodology.

Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

P_Bartell — Tue, 13 Jul 2021 15:21:09 GMT

To complement @Mark_Bailey 's advice, you may want to watch this on demand webinar I presented during my tenure as a JMP Senior Systems Engineer. It's a bit dated...I think I used JMP Pro v14...so there may some cool new capabilities in JMP Pro v16 if you are using that version? But hopefully watching the webinar will at least get you started? I do not cover principal components regression.

Partial Least Squares...What to Use When Ordinary Least Squares Regression Won't Work

Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

Nazarkovsky — Tue, 13 Jul 2021 15:48:11 GMT

That's right, thanks!
Before posting this subject, the respective tutorial had been learned and, as said, no particular info about PCR and PLSR using PCA was found.

Please, correct me, if I am wrong: if certain principal components (PCs) are explored and saved as columns, they can be used as variables for building linear regression by means of the standard "Fit Model" option, right? Or there are some specific manipulations while PCR/PLSR is arranged?
Thanks!

Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

Mark_Bailey — Tue, 13 Jul 2021 19:00:46 GMT

Yes, that is the basic idea. You are replacing a larger set of collinear measurements with a small set of orthogonal components. The components are selected based on the size of the eigenvalues and correlations in the eigenvectors. The regression analysis will determine which components, if any, are correlated with the response. In this way, if any of the components are selected for the regression model, then all the original measurements are used.

You have to study the PCA carefully. The components might not be interpretable, in which case you might get a 'good' regression but it might be difficult to interpret.. You are trading the problems of dimensionality and collinearity for the problem of interpretability. You can't 'eat your cake, and have it, too.'

Re: Principal Components Regression (PCR) and Partial Least Square Regression based on PCA

Nazarkovsky — Tue, 13 Jul 2021 19:19:53 GMT

That's great! My concern consisted basically in the technical issue of the PCR performance within JMP Pro. I totally agree that it is a delicate question about well-fitting regression and explanatory character of the involved principal components.

Many thanks!