cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • Sign-in to the JMP Community will be unavailable intermittently Dec. 6-7 due to a system update. Thank you for your understanding!
  • We’re retiring the File Exchange at the end of this year. The JMP Marketplace is now your destination for add-ins and extensions.
  • JMP 19 is here! Learn more about the new features.

Discussions

Solve problems, and share tips and tricks with other JMP users.
Choose Language Hide Translation Bar
andrey
Level II

Data Filter in JSL script running slow

I am trying to create a JSL script that will utilize data filter and select ~180 tests of interest in "ITEM_NAME" column and create a new table. When I use Data Filter and run it manually it doesn't have any delays. Though when I try to automate the process it takes ~15 minutes to create the new table. What can I do to make it run more efficiently?

Open( "$DOCUMENTS/test.jmp" );

ref10 = Data Table( "ABC" ) << Select Where(
	:ITEM_NAME == "test1" | 
	:ITEM_NAME == "test2" | 
	:ITEM_NAME =="test3" | 
	:ITEM_NAME ==
	...
	"test180"
	
) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
2 ACCEPTED SOLUTIONS

Accepted Solutions
txnelson
Super User

Re: Data Filter in JSL script running slow

Try this

Open( "$DOCUMENTS/test.jmp" );

items={"test1" "test2""test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(ITEMS) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
Jim

View solution in original post

jthi
Super User

Re: Data Filter in JSL script running slow

It is a bit difficult to understand why your script would be that slow without seeing/understanding your data (it shouldn't be).

 

You could try using << Get Rows Where and Subset instead of selecting those rows. In this example I have tried to replicate Data View by using linked subset and Untitled output table name

Names Default To Here(1); 

dt = open("$SAMPLE_DATA/Big Class.jmp");

items = {12, 13, 14, 15};
rows_of_interest = dt << get rows where(Contains(items, :age));

dt_subset = dt << Subset(Rows(rows_of_interest), Selected Columns(0), Link to original data table(1), Output Table("Untitled"));
-Jarmo

View solution in original post

7 REPLIES 7
txnelson
Super User

Re: Data Filter in JSL script running slow

Try this

Open( "$DOCUMENTS/test.jmp" );

items={"test1" "test2""test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(ITEMS) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
Jim
andrey
Level II

Re: Data Filter in JSL script running slow

Thank you Jim. For some reason I am getting the error (likely syntax related). 

andrey
Level II

Re: Data Filter in JSL script running slow

Seems to be running faster now and took around 3-4 minutes to complete but nowhere near as quick if I was to do this manually. Had to make a slight change to the code to avoid the error:

Open( "$DOCUMENTS/test.jmp" );

items={"test1", "test2", "test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(items, ITEM_NAME) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);

 

txnelson
Super User

Re: Data Filter in JSL script running slow

My error.....it was late at night....

Jim
jthi
Super User

Re: Data Filter in JSL script running slow

It is a bit difficult to understand why your script would be that slow without seeing/understanding your data (it shouldn't be).

 

You could try using << Get Rows Where and Subset instead of selecting those rows. In this example I have tried to replicate Data View by using linked subset and Untitled output table name

Names Default To Here(1); 

dt = open("$SAMPLE_DATA/Big Class.jmp");

items = {12, 13, 14, 15};
rows_of_interest = dt << get rows where(Contains(items, :age));

dt_subset = dt << Subset(Rows(rows_of_interest), Selected Columns(0), Link to original data table(1), Output Table("Untitled"));
-Jarmo
andrey
Level II

Re: Data Filter in JSL script running slow

Thank you Jarmo. You solution shaved the processing time to about 1:35 seconds.

The JMP table I am using has 13 columns and 13127825 raws. The output file has 1274416 raws.

jthi
Super User

Re: Data Filter in JSL script running slow

There are most likely optimizations which could be done but if that speed is enough for your application, I wouldn't worry about those.

-Jarmo

Recommended Articles