Choose Language Hide Translation Bar
mad
mad
Community Trekker

Concatenate Multiple Data Tables [SOLVED]

I have 605 data files I need to concatenate after filtering out the data I don't need from each file. Looping through each file, deleting the rows I don't need, concatenating the current file to the first file, then closing the current file works fine for the first 10+ files but gets exponentially slower with each additional file. Any suggestions?

I have noticed that each time a file is concatenated the source script gets added to the data table, could this be what is slowing things down? Is there a way to prevent this or delete them?

Message was edited by: MAD
0 Kudos
5 REPLIES 5
mad
mad
Community Trekker

Re: Concatenate Multiple Data Tables [SOLVED]

Each time a file is opened and after each concatenation operation add:

dt << Delete Table Property( "Source" );

This dramatically improved the speed.
0 Kudos
chungwei
Staff

Re: Concatenate Multiple Data Tables [SOLVED]

what version of JMP are you running ?
0 Kudos
mad
mad
Community Trekker

Re: Concatenate Multiple Data Tables [SOLVED]

JMP 8.0
0 Kudos
chungwei
Staff

Re: Concatenate Multiple Data Tables [SOLVED]

I think this was fixed in 8.0.1.
0 Kudos
mad
mad
Community Trekker

Re: Concatenate Multiple Data Tables [SOLVED]

You are correct. I installed the 8.0.2 update and removed my workaround. It works fine. Thank you for pointing me to the update.
0 Kudos