Selecting specific columns and concatenating

[#1]

Hi All,

Right now, I open a bunch of data tables and put it in a list called datalist.

For(i = 2 ,i <= N Items(datalist) , i++,
fdt = datalist[1];
fdt<< Concatenate(Data Table(datalist[i]),append to first table(1));
close(Data Table(datalist[i]),nosave);
);
 fdt << set name( "Final_Table");

This concatenates everything and it takes a lot of time when I try to concatenate 100's of tables together.

I have index of a column and I know exactly what I want to concatenate.

Right now, I need to concatenate first 7 columns and index number(say param_index=10) specified( column 10) for all the tables.

So the final table will just have 8 columns with all data tables.

I have been stuck on this for a while. Any idea how to do this?

OR

Is there a way to subgroup the columns (8 columns) and save it in the list in the first place, that it can be concatenated using the above script?

Thank you so much.

[#2]

Hello @vishwasanj, 
      Try this 

Clear Globals() ; Clear Log(); 

// Open Sample Data 
dt = Open( "$SAMPLE_DATA/Air Traffic.jmp" );

// Generate Some random tables 
nSteps = N Rows(dt)/100; 

// Define Columns you want to use  
DesiredCols = {"Airline","Flight Number","Flight ID","Event"};
for( i = 1 , i < = N Row(dt), i++,
		dt << Select Randomly( 0.1 );
		dt1 = dt << Subset(Selected Rows(1), Columns(DesiredCols)); 
		dt1 << Show Window(0); 
   );

This shows how to select only the columns you want. Once you have this - you can use the code you have to concatenate only the columns you needed. 

Best 

Uday 

[#3]

I would suggest that as you open up each table, that you delete the unneeded columns.  I might also make all but the first table "Private".

You may also see if doing the concatenation of all of the tables in one step rather than looping through each table and concatenating it, and then moving on to the next table.  This may prove to be faster.

[#4]

Building on Jim's thoughts, perhaps the 'Select Columns' option for 'Open()' might also be useful.