topic Efficient formula in Israel JMP Users - קבוצת משתמשים - ישראל

Efficient formula

Idokol — Thu, 31 Jan 2019 06:02:19 GMT

Hey,

I have big data table (~30million rows*20 columns) and this Pass/Fail test makes JMP stuck.

The PC is Windows 10 64-bit, i7-4770 @ 3.4Ghz, SSD and 32GB DDR3 1600Mhz.

The JSL is:

cctable << New Column( "New status", Character, Nominal, Formula( If( :result < :L | :result > :H, "Fail", "Pass" ) ) );

Can it be more efficient? Is there's a better choise in JMP to do it?

Thanks,

Ido

Re: Efficient formula

tom_abramov — Thu, 31 Jan 2019 07:14:06 GMT

Hi,

Don't use column formulas with such a table.

Try to run a calculation with JSL and paste the results.

Please see attached a table and a script I used.

Regards,

Tom.

Re: Efficient formula

Idokol — Thu, 31 Jan 2019 15:23:15 GMT

Thanks, but the "New Column( ... )" command takes time and everything get's stuck as well.

I will try to use update/join

If you (or anyone else :) ) have a nother idea- I will glad to hear it.

Thanks.

Re: Efficient formula

Craige_Hales — Fri, 01 Feb 2019 11:46:42 GMT

My test takes 15 seconds to create the table and 15 seconds to add the extra column. I am using <<runFormulas to make the formula evaluate immediately rather than waiting for idle time.

start = Tick Seconds();
dt = New Table( "Untitled",
	Add Rows( 30e6 ),
	New Column( "L", Numeric, "Continuous", Format( "Best", 12 ), Formula( Random Integer( 1, 9 ) ) ),
	New Column( "result", Numeric, "Continuous", Format( "Best", 12 ), Formula( Random Integer( 1, 9 ) ) ),
	New Column( "H", Numeric, "Continuous", Format( "Best", 12 ), Formula( Random Integer( 1, 9 ) ) )
);
dt<<runformulas;
stop = Tick Seconds();
Show("create", stop - start, N Rows( dt ) );

start = Tick Seconds();
dt << New Column( "New status", Character, Nominal, Formula( If( :result < :L | :result > :H, "Fail", "Pass" ) ) );
dt << runformulas;
stop = Tick Seconds();
Show( "add", stop - start, N Rows( dt ) );

"create";
stop - start = 12.8833333333605;
N Rows(dt) = 30000000;
"add";
stop - start = 12.6833333332906;
N Rows(dt) = 30000000;

Formula added

Re: Efficient formula

Idokol — Wed, 06 Feb 2019 12:52:57 GMT

Thanks!
I will do it.

Although what that helped the most was table of 1M * 100 rather then 80M * 5