Subscribe Bookmark RSS Feed

Re: Selecting specific columns and concatenating

vishwasanj

Contributor

Joined:

Dec 27, 2016

Hi All,

 

Right now, I open a bunch of data tables and put it in a list called datalist.

 

For(i = 2 ,i <= N Items(datalist) , i++,
fdt = datalist[1];
fdt<< Concatenate(Data Table(datalist[i]),append to first table(1));
close(Data Table(datalist[i]),nosave);
);
 fdt << set name( "Final_Table");

 

 

This concatenates everything and it takes a lot of time when I try to concatenate 100's of tables together.

I have index of a column and I know exactly what I want to concatenate.

Right now, I need to concatenate first 7 columns and index number(say param_index=10) specified( column 10) for all the tables.

So the final table will just have 8 columns with all data tables.

 

I have been stuck on this for a while. Any idea how to do this?

 

OR

 

Is there a way to subgroup the columns (8 columns) and save it in the list in the first place, that it can be concatenated using the above script?

 

 

 

Thank you so much.

 

3 REPLIES
uday_guntupalli

Community Trekker

Joined:

Sep 15, 2014

Hello @vishwasanj
      Try this 

 

Clear Globals() ; Clear Log(); 

// Open Sample Data 
dt = Open( "$SAMPLE_DATA/Air Traffic.jmp" );

// Generate Some random tables 
nSteps = N Rows(dt)/100; 

// Define Columns you want to use  
DesiredCols = {"Airline","Flight Number","Flight ID","Event"};
for( i = 1 , i < = N Row(dt), i++,
		dt << Select Randomly( 0.1 );
		dt1 = dt << Subset(Selected Rows(1), Columns(DesiredCols)); 
		dt1 << Show Window(0); 
   );

This shows how to select only the columns you want. Once you have this - you can use the code you have to concatenate only the columns you needed. 

 

Best 

Uday 

Best
Uday
txnelson

Super User

Joined:

Jun 22, 2012

I would suggest that as you open up each table, that you delete the unneeded columns.  I might also make all but the first table "Private".

You may also see if doing the concatenation of all of the tables in one step rather than looping through each table and concatenating it, and then moving on to the next table.  This may prove to be faster.

Jim
Highlighted
ian_jmp

Staff

Joined:

Jun 23, 2011

Building on Jim's thoughts, perhaps the 'Select Columns' option for 'Open()' might also be useful.