topic Re: Copy column subset or cell range from table to table in Discussions

Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 07:26:49 GMT

I have two data tables. Some columns on table2 are identical to columns table1.

I wasnt to copy the data of the identical columns from table2 to table1.

That means, of course, that (1) table1 will be expanded with the number or rows in table2 and, (2) there will be missing data in table1. That's OK.

My issue is how to copy the data from table2 to table1.

E.g. - both tables have column "NAME". Table1 has 1000 rows, table2 has 2000 rows.

The results will be: table1 with 3000 rows, and "NAME" on table1 will be a "concatenation" of both NAME columns. Other columns in table1 will have 2000 missing/empty data.

BTW, in reality, tables are much larger and there are 10-20 identical columns (out of ~100). I will deal with it once I figure out the basic function. I mention it in case there's an issue with such size.

Many thanks

Re: Copy column subset or cell range from table to table

txnelson — Wed, 11 Jul 2018 10:17:51 GMT

JMP will concatenate data tables using

Tables==>Concatenate

It will line up the columns by column name when it concatenates. Missing values will automatically be added in the rows for columns that do not appear in one or the other data table.

The Concatenation Platform keeps the Name in tact for matching columns, and just adds new column names for columns that were not in the base data table.

For documentation, see:

Help==>Books==>Using JMP

Re: Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 11:12:11 GMT

Thanks. I did thought about it. However, I need to do it with a script because I have many such files and the order of the columns changes from one to one.

I'd like to figure out some code that'll do it.

Thanks

Re: Copy column subset or cell range from table to table

txnelson — Wed, 11 Jul 2018 11:23:21 GMT

You can have JMP generate the code for you.

Run the

Tables==>Concatenate

interactively, and then in the new data table, go into the Tables Panel on the left side of the data table and right click on the "Source" entry, and select "Edit". There you will see the JSL required to concatenate the files together.

The JSL is basically

dt1=data table(<your first table>);
dt2=data table(<your second table>);

dt1 = dt1 << concatenate( dt2, Append to First Table(1));

Re: Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 11:35:11 GMT

Thanks.

Concatenate function won't do because I need to do it only for columns that are identical in both tables, skipping all others.

I tried storing the source column data in a matrix/list and than setting the valuse on the target column (the second table) using subscription [I,j] but I guess I'm missing something.

Re: Copy column subset or cell range from table to table

txnelson — Wed, 11 Jul 2018 11:43:10 GMT

Concatenation does not egnore non matching columns. Here is a sample using the Big Class and Big Class Families, sample data tables. Please note, that the Big Class data table ends up will all of the columns that the Big Class Families has that are not in the original Big Class data table.

Names Default To Here( 1 );
dt1 = Open( "$SAMPLE_DATA/big class.jmp" );
dt2 = Open( "$SAMPLE_DATA/big class families.jmp" );

dt1 << concatenate( dt2, Append to First Table( 1 ) );

Re: Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 14:41:11 GMT

Thanks again. Unfortunately, concatenate won't do due to the bizzar nature of my data sets.

I need to concatenate only certain columns, ignoring the test.

I can always loop through all rows in the source table and insert rows in the target tables and than set column value one by one. But I'm afraid it'll hog CPU - my script will process over 100 files with dozens of columns and thousands of rows. Besides, it's not an elegant solution, albeit probably inevitable.

I'll simplify my question.

Can I do something like that?

Column(dt, i)[from,to] = list or matrix of values

Thanks for you patience,

Re: Copy column subset or cell range from table to table

Jeff_Perkinson — Wed, 11 Jul 2018 14:52:26 GMT

Don't give up on the builtin functions, like Tables->Concatenate so quickly.

If you only want the columns that have the same name between two tables, you can use Tables->Subset to create two tables with only those columns from each table and then concatenate those two tables.

Re: Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 15:04:11 GMT

OK. Good encouragement.

Table1 has columns a b c d e, table2 has columns x y e z a u v. I want to concatenate to table1 only columns e a from table2. How would you go about doing that?

If I understand you, I need to create table2.1 with columns e a, than reorder them to be a e, and concatenate table2.1 to table1?

Re: Copy column subset or cell range from table to table

Jeff_Perkinson — Wed, 11 Jul 2018 15:31:43 GMT

@RonB wrote:

If I understand you, I need to create table2.1 with columns e a, than reorder them to be a e, and concatenate table2.1 to table1?

You've got it, no need to reorder the columns though. Tables->Concatenate matches on column name, not position.

Subset relies on column selection, so you need to select the columns that are in common between the two tables. But that's easy because you can just get all the column names from table1 and select that whole list against table2. Any that don't exist will get ignored.

Here's an example using two of the sample data tables.

bc = Open( "$SAMPLE_DATA\Big Class.jmp" );
fit = Open( "$SAMPLE_DATA\Fitness.jmp" );

fit_cols = fit << get column names;

bc << select columns( fit_cols );

sub_bc = bc << Subset( All rows, Selected columns only( 1 ) );

fit << Concatenate( sub_bc );

//if you don't want a new table you can use "Append to first table"

fit << Concatenate( sub_bc, Append to first table );

Re: Copy column subset or cell range from table to table

RonB — Wed, 11 Jul 2018 16:11:11 GMT

I'll give it a try and report back (may take a while due to unrelated circumstances).

Many tx

Re: Copy column subset or cell range from table to table

RonB — Thu, 12 Jul 2018 14:41:24 GMT

Worked. Many thanks. Now I just have to figure out how to copy the non-matching columns but I think I can manage.

Re: Copy column subset or cell range from table to table

Jeff_Perkinson — Thu, 12 Jul 2018 14:55:08 GMT

@RonB wrote:

Now I just have to figure out how to copy the non-matching columns but I think I can manage.

Non-matching columns from which table? The example above keeps all the columns from the Fitness table and concatenates only the columns that have the same name from Big Class. It does this by only subsetting Big Class to get the columns with the same names and then concatenating that to Fitness (which still has all of its columns). There shouldn't be any work to keep the non-matching columns.

Re: Copy column subset or cell range from table to table

RonB — Thu, 12 Jul 2018 15:35:11 GMT

Yes, I understand that. But the data structure I'm dealing with requires to copy also columns in Big Class that do not have the same name in Fitness. I.e., these columns will be "vertically concatenated" to the output table as new columns. These columns will begin with missing data (as many rows as are in Fitness) followed by the data from Big Class.
I think I can do it by first adding to Fitness new columns with all the names in Big Class that are different. Than use the same method you showed me, subsetting Big Class with only these columns and doing the concatenation.
It's a lot of work but I see no other way to deal with the large number of tables and columns I have.
Thanks yet again

Re: Copy column subset or cell range from table to table

Jeff_Perkinson — Thu, 12 Jul 2018 16:39:16 GMT

@RonB wrote:
Yes, I understand that. But the data structure I'm dealing with requires to copy also columns in Big Class that do not have the same name in Fitness. I.e., these columns will be "vertically concatenated" to the output table as new columns. These columns will begin with missing data (as many rows as are in Fitness) followed by the data from Big Class.

How is this different from simply concatenating the two tables together to begin with? Doing that you'll end up with a table with all the rows and all the columns from both tables. The columns that don't have matching names in both tables will be empty for the rows that didn't have those columns originally.

Let's stop talking abstractly and be more concrete. Here are two tables:

Table1

A	B	C
1	1	1
2	2	2
3	3	3

Table2

A	B	D
4	4	4
5	5	5
6	6	6

Using Tables->Concatenate on these two tables, you'll end up with this table.

Table3

A	B	C	D
1	1	1	•
2	2	2	•
3	3	3	•
4	4	•	4
5	5	•	5
6	6	•	6

The way I interpret your description above makes me think that's the table you want. If it's not, I'm sorry I've misunderstood your requirement.

Re: Copy column subset or cell range from table to table

RonB — Thu, 12 Jul 2018 18:28:11 GMT

Something didn't work for me with the example.
But, never mind, I got the idea and it seemed to work with running a test on few input files..
Seems like I complicated things unnecessarily while a straight forward way was under my nose.
Thanks for your patience