JMP v19 Graph Builder performance with large tables (13M+ rows)

[#1]

I’m running into significant performance issues with Graph Builder in JMP v19 when working with a large table (~13 million rows). In JMP v18, I was able to interact fluidly with the same dataset — panning, zooming, and switching variables were all manageable.  It’s a single table of about 13M rows, with a couple-dozen columns, on a pretty heavy workstation with 64 GB RAM.

In JMP v19, however, I’m seeing frequent “hourglass” delays and “JMP is not responding” messages whenever I try to create or adjust plots. Going back to JMP v18 with the same data and settings works smoothly again.

I’m wondering if anyone else has noticed degraded performance in Graph Builder with large tables since upgrading to JMP v19. If so, it might be worth JMP taking a look at what changed between versions.

Has anyone found any workarounds or settings that help (e.g., memory preferences, GPU settings, etc.)?

[#2]

1.3 million rows here. My impression is GB responds sluggishly both, in JMP 18 and 19.

[#3]

In the past, I experienced severe performance issues, which I could often trace back to unexpected recalculations of column formulas.
a severe issue which got fixed back in JMP18:
Why does selecting/deselecting rows trigger column formulas to be re-evaluated? 


Some help: In versions of JMP up to and including 18, such recalculations are indicated in the status bar.
In JMP 19, if you want to be notified of a recalculation, you can ask JMP to inform you:

https://community.jmp.com/t5/Discussions/Undocumented-Secrets/m-p/908936/highlight/true#M106749 



If this is not the reason for the bad performance, you could check the memory consumption. If the memory is shared between programs and the graphics card, limited memory could lead to severe effects.

To dig deeper, could you create a dummy file for us to analyze?
Sometimes, the issue is specific to a certain type of data or plot.

No argument why something changed in JMP19 - but I agree with @Ressel :
For many types of Plots, some Mio rows is where JMP starts to respond sluggish.
JMP can handle much larger data files, but at some time the user doesn't want to wait.
If 0.5s is OK to wait, 5x 0.5s feels much less comfortable. So, e.g. if 1 Mio rows feels smooth, 5Mio rows is no fun.

[#4]

I don't know if this helps, but I'm running JMP 18 on a 16GB machine.  I just tested Graph Builder on a file with 26 columns and 3.5 million rows and it worked quickly with no delays.  

[#5]

I think the number of columns doesn't affect the speed.
And for the rows, the time is different for the calculation of a bar graph of the mean and a point plot of all individual data points.

e.g. on my old notebook it takes ~ 6 seconds to calculate a box plot of sin() compared to 2 seconds for a box plot of row() - and quite comparable time in JMP 18 and JMP19.

Calculating a point plot takes longer, and it seems: a bit longer with JMP19 than with JMP18 ?
How about your system?

New Table( "test",
	Add Rows( 10000000 ),
	New Table Variable( "categories", 20),
	New Column( "row",
		Formula( Row() )
	),
	New Column( "sin",
		Formula( Sin( :row / 10 ) )
	),
	New Column( "rnd",
		Formula( Random Integer( :categories ) )
	)
)

[#6]

5'' for all columns to be populated on my machine, give or take.

runtime of the JSL itself is <0.1''.

Is this good? I mean, 10 Mio rows, I think it is very much acceptable. However, dropping these columns into GB, it takes a while for it to respond. Still much better than trying to solve an issue like this in (dare I name it) Excel.

[#7]

roundabout & acceptable.

On the other hand, just imagine how many calculations it takes to render a graphics game in real time.

Next step: create a graph. What is smooth, without no delay, what is too much waiting time?

To get more precise values for the creation of the table, one can use hptime().
and to take the calculation of the column formulas into account, one can wait for dt << run formulas.

t0 = hp time ();
dt = New Table( "test",
	Add Rows( 10000000 ),
	New Table Variable( "categories", 20),

);

	New Column( "row",
		Formula( Row() )
	);
	New Column( "sin",
		Formula( Sin( :row / 10 ) )
	);
	New Column( "rnd",
		Formula( Random Integer( :categories ) )
	);

dt << run formulas;

write("t(s): ", (hptime()-t0)/1000000)