Save to Database is too slow (10k records per minute) has a reply by msharp that reminded me of a performance enhancement scheduled for JMP 13. (Thanks Jeff Polzin!)
Msharp mentions that the database update function is faster than another technique, but still not great. (And see bryan.boone answer to the original question as well...another JMP 13 performance enhancement.) I looked at the script (which is a clever way to use JSL and SQL) and saw it adds items to the end of a list, something like this:
// build forward
forwardList = {};
starttime = Tick Seconds();
For( i = 0, i < 30000, i++,
Insert Into( forwardList, i )
);
stoptime = Tick Seconds();
Show( stoptime - starttime );
stoptime - starttime = 1.68333333334886; // JMP 12 time
that's really slow in JMP 12 and earlier. The list is held internally by its first item, and the last item must be discovered by walking the length of the list. InsertInto() adds to the end of the list by default. That leads to N^2 performance: doubling the length of the list quadruples the time. There is a way to avoid it:
// build reverse
reverseList = {};
starttime = Tick Seconds();
For( i = 0, i < 30000, i++,
Insert Into( reverseList, i, 1 )
);
reverseList = Reverse( reverseList );
stoptime = Tick Seconds();
Show( stoptime - starttime );
stoptime - starttime = 0.0333333334419876;
that's better. The insertinto() function is called with a 3rd argument of 1 to insert at the front of the list. Notice the call to the reverse() function is also accounted for in the reported time. And the answer is the same:
show( reverseList == forwardList);
reverseList == forwardList = 1; // good, same answer either way
Looking forward to JMP 13? here it is:
// build forward
forwardList = {};
starttime = Tick Seconds();
For( i = 0, i < 30000, i++,
Insert Into( forwardList, i )
);
stoptime = Tick Seconds();
Show( stoptime - starttime );
stoptime - starttime = 0.0166666668374091; // Jeff is amazing!
Long story short, if you have a JMP 12 script that is building a list of more than 100 or so items, insert at the front, then reverse. JMP 13 will be fast either way.
-Jeff
