cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • We’re improving the Learn JMP page, and want your feedback! Take the survey
  • JMP monthly Newswire gives user tips and learning events. Subscribe
Choose Language Hide Translation Bar
andrey
Level II

Data Filter in JSL script running slow

I am trying to create a JSL script that will utilize data filter and select ~180 tests of interest in "ITEM_NAME" column and create a new table. When I use Data Filter and run it manually it doesn't have any delays. Though when I try to automate the process it takes ~15 minutes to create the new table. What can I do to make it run more efficiently?

Open( "$DOCUMENTS/test.jmp" );

ref10 = Data Table( "ABC" ) << Select Where(
	:ITEM_NAME == "test1" | 
	:ITEM_NAME == "test2" | 
	:ITEM_NAME =="test3" | 
	:ITEM_NAME ==
	...
	"test180"
	
) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
2 ACCEPTED SOLUTIONS

Accepted Solutions
txnelson
Super User

Re: Data Filter in JSL script running slow

Try this

Open( "$DOCUMENTS/test.jmp" );

items={"test1" "test2""test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(ITEMS) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
Jim

View solution in original post

jthi
Super User

Re: Data Filter in JSL script running slow

It is a bit difficult to understand why your script would be that slow without seeing/understanding your data (it shouldn't be).

 

You could try using << Get Rows Where and Subset instead of selecting those rows. In this example I have tried to replicate Data View by using linked subset and Untitled output table name

Names Default To Here(1); 

dt = open("$SAMPLE_DATA/Big Class.jmp");

items = {12, 13, 14, 15};
rows_of_interest = dt << get rows where(Contains(items, :age));

dt_subset = dt << Subset(Rows(rows_of_interest), Selected Columns(0), Link to original data table(1), Output Table("Untitled"));
-Jarmo

View solution in original post

7 REPLIES 7
txnelson
Super User

Re: Data Filter in JSL script running slow

Try this

Open( "$DOCUMENTS/test.jmp" );

items={"test1" "test2""test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(ITEMS) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);
Jim
andrey
Level II

Re: Data Filter in JSL script running slow

Thank you Jim. For some reason I am getting the error (likely syntax related). 

andrey
Level II

Re: Data Filter in JSL script running slow

Seems to be running faster now and took around 3-4 minutes to complete but nowhere near as quick if I was to do this manually. Had to make a slight change to the code to avoid the error:

Open( "$DOCUMENTS/test.jmp" );

items={"test1", "test2", "test3"..........};

ref10 = Data Table( "ABC" ) << Select Where(
	contains(items, ITEM_NAME) ) << Data View;

Data Table( ref10 ) << Save(
	"$DOCUMENTS/filtered.jmp"
);

 

txnelson
Super User

Re: Data Filter in JSL script running slow

My error.....it was late at night....

Jim
jthi
Super User

Re: Data Filter in JSL script running slow

It is a bit difficult to understand why your script would be that slow without seeing/understanding your data (it shouldn't be).

 

You could try using << Get Rows Where and Subset instead of selecting those rows. In this example I have tried to replicate Data View by using linked subset and Untitled output table name

Names Default To Here(1); 

dt = open("$SAMPLE_DATA/Big Class.jmp");

items = {12, 13, 14, 15};
rows_of_interest = dt << get rows where(Contains(items, :age));

dt_subset = dt << Subset(Rows(rows_of_interest), Selected Columns(0), Link to original data table(1), Output Table("Untitled"));
-Jarmo
andrey
Level II

Re: Data Filter in JSL script running slow

Thank you Jarmo. You solution shaved the processing time to about 1:35 seconds.

The JMP table I am using has 13 columns and 13127825 raws. The output file has 1274416 raws.

jthi
Super User

Re: Data Filter in JSL script running slow

There are most likely optimizations which could be done but if that speed is enough for your application, I wouldn't worry about those.

-Jarmo

Recommended Articles