Our World Statistics Day conversations have been a great reminder of how much statistics can inform our lives. Do you have an example of how statistics has made a difference in your life? Share your story with the Community!
Choose Language Hide Translation Bar
Level I

Memory usage when creating many tables

When creating many tables and then closing them JMP seems to keep the tables or some remnants in memory. The memory is sometimes released after a while. I ran into this when simulating a measurement situation with noise sources. JMP din't crash but after RAM ran out windows started swapping to HDD.


Below is the simplest example where I was able to repeat the problem.


Question: is this expected behaviour and/or did I do something funny?


Note to anyone encountering the same: uncommenting the Wait(0); removes the problem, but seems to have a small (~10 %) performance penalty at least in the case where this was a problem.


For (i = 1, i <= 100000, i++,
	dt = New Table("Name of the table",
		New Column("Name of the column", Set Values(1::20000)),
	// Do stuff
	Close(dt, "No Save");	

JMP 14.0.0 64-bit, Win 10

Level VIII

Re: Memory usage when creating many tables

         Try to include an "invisible" or a "private" flag when you open so many data tables. Making Data Tables "invisible" will help you see some improvement and making it "private" will help you see significant improvements. 




Re: Memory usage when creating many tables

Why create a new table in each iteration? You could create a data table before iterating and the update the contents in each iteration.

Learn it once, use it forever!
Level I

Re: Memory usage when creating many tables

Mostly because I used parts from scripts I had already written before and didn't want to change them. In many cases I prefer slow scripts over working on the script to make it faster.


In some cases the data tables get their data from matrices:

dt = As Table(matrixWithData, private);

Is there a way to easily put the data into an existing data table? That would make the changes relatively simple.


Re: Memory usage when creating many tables

Regarding the last question, you could use data table subscripting:


// Make a starting table to update later
mat = J(10, 5, RandomNormal());
dt = AsTable(mat);
dt << setName("My Table");

// Update the last four columns of dt (using data table subscripting) with new values
mat = J(10, 4, RandomNormal());
dt[1::10, 2::5] = mat;
Article Labels

    There are no labels assigned to this post.