topic Re: Storing data tables as metadata (table variables) JSL in Discussions

Storing data tables as metadata (table variables) JSL

FN — Fri, 09 Jun 2023 16:08:26 GMT

Table variables are used to store metadata within a data table (JSL doc, general doc)

However, they seem to be limited to variables as strings only (see code below).

What would be the best option to store a data table as a table variable?

Names Default To Here( 1 );

dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

dt_metadata = Open( "$SAMPLE_DATA/Solubility.jmp" );

dt << Set Table Variable( "dt_user", dt_metadata );

close(dt_metadata);

dt_recovered = dt << Get Table Variable("dt_user");

show(dt_recovered);
show(Is String(dt_recovered));

//dt_recovered = "Solubility";
//Is String(dt_recovered) = 1;

Re: Storing data tables as metadata (table variables) JSL

Craige_Hales — Wed, 19 Apr 2023 15:26:13 GMT

https://community.jmp.com/t5/Discussions/difference-between-set-and-new-table-variable-strange-results-in/m-p/429212/highlight/true#M67868 from describes four different functions, set/get/new table variable and dt:tablevar and recommends using set table variable and dt:tablevar.

Re: Storing data tables as metadata (table variables) JSL

FN — Fri, 21 Apr 2023 11:11:59 GMT

The variable to store is a data table. The dt:tablevar doesn't seem to work in this case.

Re: Storing data tables as metadata (table variables) JSL

Jeff_Perkinson — Fri, 21 Apr 2023 15:12:45 GMT

Table variables are character strings.

If you want dt_recovered to be the same after you'll need to convert dt_metadata into the character string representation of the expression when you store it in the table variable. You'll then need to Parse it when you get it back out of the variable.

Names Default To Here( 1 );

dt = Open( "$SAMPLE_DATA/Big Class.jmp" );

dt_metadata = Open( "$SAMPLE_DATA/Solubility.jmp" );

dt << Set Table Variable( "dt_user", char(dt_metadata));

close(dt_metadata);

dt_recovered = parse(dt << Get Table Variable("dt_user"));

show(dt_recovered);
show(Is String(dt_recovered));

//dt_recovered = Data Table("Solubility.jmp");

//Is String(dt_recovered) = 0;

Re: Storing data tables as metadata (table variables) JSL

Craige_Hales — Sat, 22 Apr 2023 13:04:56 GMT

Here are three examples that embed the data for table B in table A . I prefer the dtb3 example for reasonable sized tables. The dtb4 example is what I think you described; it is maybe better for a huge B table, but it is very opaque to anyone else. dtb5 combines the two ideas.

// Make sample tables. When done, aa will have two table scripts and a table variable.
// the scripts and variables show three ways to embed bb in aa.
dta = New Table( "aa", New Column( "a", Numeric, "Continuous", Format( "Best", 12 ), Set Values( [1] ) ) );
dtb = New Table( "bb", New Column( "b", Numeric, "Continuous", Format( "Best", 12 ), Set Values( [2] ) ) );

// dtb3: I think you should do this with a table script because it can be
// opened from the GUI or from JSL. The downside is if the table is
// very large it will be slow to save and slow to open because it is
// making a script to recreate the table.
Eval( Eval Expr( dta << NewScript( "bbb", Expr( dtb << getScript ) ) ) );
// jsl to open:
dtb3 = Eval( dta << getScript( "bbb" ) );
dtb3 << SetName( "dtb3" ); // optional, was "bb 2"


// dtb4: it gets ugly with a table variable, there are several possibilities. This
// will save the table and grab the binary from the disk file and make a character
// string from that. You might choose to do this for a really large file, but I'm
// not sure what issues you'll run into. 
dtb << Save( "$temp/bb.jmp" );
blob = Load Text File( "$temp/bb.jmp", blob );
blob = Gzip Compress( blob );// optional
text = Char( blob );
dta << SetTableVariable( "bbbb", text );
// reopen
text = dta:bbbb;
blob = Parse( text );
blob = Gzip Uncompress( blob );//optional
dtb4 = Open( blob, "jmp" );
dtb4 << SetName( "dtb4" ); // optional, was "untitled"


// dtb5: you could combine the two ideas, saving the binary data as a table script so
// it can be used from the GUI
dtb << Save( "$temp/bb.jmp" );
blob = Load Text File( "$temp/bb.jmp", blob );
blob = Gzip Compress( blob );// optional
text = Char( blob );
Eval(
	Eval Expr(
		dta << NewScript(
			"bbbbb",
			Local( {text, blob, dt},
				text = Expr( text );
				blob = Parse( text );
				blob = Gzip Uncompress( blob );//optional
				dt = Open( blob, "jmp" );
				dt;
			)
		)
	)
);
// reopen 
dtb5 = Eval( dta << getScript( "bbbbb" ) );
dtb5 << SetName( "dtb5" ); // optional, was "untitled"

Re: Storing data tables as metadata (table variables) JSL

Craige_Hales — Sat, 22 Apr 2023 13:50:38 GMT

Or, if you mean to leave a link rather than embedding the data, perhaps use the file name for the linked table:

dta = New Table( "aa", New Column( "a", Numeric, "Continuous", Format( "Best", 12 ), Set Values( [1] ) ) );
dtb = New Table( "bb", New Column( "b", Numeric, "Continuous", Format( "Best", 12 ), Set Values( [2] ) ) );
dta << Save( "$temp/aa.jmp" );
dtb << Save( "$temp/bb.jmp" );

// leave table B's filepath in table A

dta << SetTableVariable("path to B", dtb<<getpath);

close(dtb);

dtb2 = open(dta:path to B); // apparently case matters with table variable names!

Re: Storing data tables as metadata (table variables) JSL

hogi — Mon, 01 Jul 2024 16:25:55 GMT

For columns a user can specify column properties - and there is even a possibility to define user-defined properties.
How about a similar framework for tables?

Here is a wish :