Choose Language Hide Translation Bar
AlotaibiA
Occasional Contributor

Select column from effect summary

Hello everyone:

 

I have a huge dataset with 1000+ columns.  I am doing a regression model using Fit model function.  The effect summary gave me 171 columns out of 6700+ which are the best for my model.  My question is how can I choose those 171 easily without picking them one by one? please see the picture below for more details.  

Screen Shot 2019-12-01 at 9.47.34 AM.png

 

 

Thanks,

 

Anwar

0 Kudos
5 REPLIES 5

Re: Select column from effect summary

You can choose anything in the Effect Summary list by clicking. You can click and drag over the items your want or click one end and shift-click the other end.

Learn it once, use it forever!
julian
Staff

Re: Select column from effect summary

Hi @AlotaibiA,

Just to add, once you follow @markbailey's instructions for selecting all of your columns of interest, you can copy those columns for use in other platform launch windows by right-clicking > Copy Columns. Then, in any launch window, you can right-click > paste into a column role, as in the example below. You can also paste those columns into a script window (or any document) for use later.

 

copypaste.gif

 

I hope this helps!

@julian  

AlotaibiA
Occasional Contributor

Re: Select column from effect summary

Thanks! How can I select them to subset them from the dataset?

0 Kudos
julian
Staff

Re: Select column from effect summary

Hi @AlotaibiA,

 

That seems to be a bit more tricky. And, I just discovered that Right Click > Copy Columns copies not just the selected columns, but ALL the columns in that table. So, here's what I would do (until we figure out a more elegant solution): rather than right-click select columns, right-click > make into data table. This will make a table of those columns (with their statistics). The columns you selected before will still be selected (as rows). Click on the "Source" column so that you have selected just the column names, and now you can use a standard edit > copy to copy just those columns to your clipboard. 

 

Now, as for selecting those columns in your original table, I do not have a simple interactive solution handy. But, we can script our way around this until we figure out one. With your original table in the foreground, you can use the following script:

 

colNames = words(Get Clipboard(),"\!r");
colList = {};

for(i=1,i<=length(colNames),i++,
	insertInto(colList, parse(":Name(\!"" || colNames[i] || "\!")"	) )
);

current data table() << clear column selection;
current data table() << Select Columns( colList );

This will take your clipboard (with those column names), and assemble a list of column references, and then send a message to your current table to select those columns. 

 

It's not the most elegant solution, but it's something to start! I hope this helps until we can come up with something simple.

 

@julian 

0 Kudos
Highlighted
mzwald
Staff

Re: Select column from effect summary

Another option is to use the stepwise personality under Fit Model to identify significant effects but you would be guaranteed in this case to end up with an overfit model because you have more model effects than rows of data (6700 model effects and 4000 rows of data).  Stepwise is primarily effective when you have relatively few model effects compared to the sample size of data.

In your particular case, you would need to use the Partial Least Squares modeling option under multivariate if your response is numeric, or the PLS option under Fit Model if your response is categorical (this option requires JMP Pro). 

 

Another option (also requires JMP Pro) is to create a Fit Model using Training/Validation/Test sets which can help mitigate overfitting.

0 Kudos