topic Re: Efficient row state coding in Discussions

Efficient row state coding

mikedriscoll — Thu, 04 May 2017 18:18:39 GMT

Hi, I'm trying to figure out the fastest way to change row states in JSL. I normally try to avoid selecting rows because it is slower than get rows where(), but I don't see how to use a matrix or list of rows to change row states. Do rows need to be selected to change the state?

Mildly related question: Is the color pallette scriptable? I see the "colorbox()" dislplay box but the scripting index says it is only accesible by the platform. I don't necessarily need it, as I was able to find a couple workarounds. An example of what I mean by this is shown below the code here.

Thanks,
Mike

//Clear Log();
Names Default To Here( 1 );
dt = open( "$SAMPLE_DATA\Big Class.jmp");

//get rows where; not sure what to do with this
myRows = dt << get rows where(:height == 63);

// use select where works but could be slower
dt << select where(:height == 63);
dt << exclude(1);
dt << colors("Red");

Color pallette, example from user preferences:

Re: Efficient row state coding

txnelson — Thu, 04 May 2017 19:57:29 GMT

I don't have an answer to your color pallette question, but I did run a quick Select Where vs. Get Rows Where and found little difference

Names Default To Here( 1 );
dt = Open( "$SAMPLE_DATA\semiconductor capability.jmp" );
start = HP Time();
For( i = 1, i <= N Rows( dt ), i++,
	dt << select where( :NPN1 > :npn1[i] )
);
Show( "select where", Format( (HP Time() - start) / 100, "Hr:m:s" ) );
start = HP Time();
For( i = 1, i <= N Rows( dt ), i++,
	dt << select where( :NPN1 > :npn1[i] )
);
Show( "get rows", Format( (HP Time() - start) / 100, "Hr:m:s" ) );

and I got the results:

"select where";= "2:51:55";
"get rows"; = "2:50:40";

Re: Efficient row state coding

mikedriscoll — Thu, 04 May 2017 20:28:15 GMT

Thanks Jim. The habit of choosing "getting" over "selecting" is kind of ingrained after years of slow script results in both vba and jsl, but I suppose I'm mostly thinking of cases where I need to iterate not only through rows but through each column.

Here are a couple of scenarios regarding semiconductor test data, where each row is a device tested in production, and each column (after the index columns) is a parametric test on the devices. Each column has spec limits stored as a property. One script loops through and determines on a column by column basis the yield, high side and low side fallout, etc (lots of other stuff but that's irrelevant). Generally with semiconductor data, it is tested with stop on first fail, so the columns to the right of a failure might be missing. If data is instead tested continue on fail, it might be nice to convert to 'stop on fail.' So I have a script to convert to stop-on-first-fail data as well. In both of these cases, changing over to get rows where() vs select where() dramatically improved script execution time. It's possible that it might be faster using select where() if the table is invisible; I haven't checked this.

For the row states though, it is just looping through once, so now that I take a step back and think about it, it probably isn't too bad.

Here's a screenshot of two script versions for my stop on fail script. This change made a huge difference in execution time.

And a whole other topic might be why I'm parsing it as a string. I'd have to go back and look at this again, but years ago I found that parsing as a string was much faster, but that might have been while I still had the select where() statements.

Re: Efficient row state coding

txnelson — Fri, 05 May 2017 00:15:05 GMT

I worked in the semiconductor industry too, and found that on many of the large applications, moving the parametric data into matrices and using the various matrix functions rather than dealing with data table functions is a much faster method.

Re: Efficient row state coding

mikedriscoll — Fri, 05 May 2017 13:43:13 GMT

Yes, I tend to forget that and get reminded from time to time when I'm reviewing JMP's online help. I probably just used the "select where()" and later get "rows where()" for convenience with respect to handling excluded rows and quickly (coding wise, not execution-wise) grabbing those that are above / below spec limits, though I suppose I could get a separate column matrix of excluded() containing ones and zeros, and just multiply. So long as it handles missing values correctly it would probably work fine and run quickly.