Insert one expression into another using Eval Insert, Eval Expr, Parse, and Substitute

[#1]

Problem

You need to insert the value of a variable into an expression without evaluating the whole code.  Examples include:

1. Writing column formulas using a script
2. Writing scripts using another script

Below is a script to transform the selected columns in a table. This does NOT work, because the expression 'cols[i]' is saved in the column formula instead of the column name.

//Open A Table and select a few columns
dt = Open( "$Sample_data/iris.jmp" );
cols = {Sepal length, Petal length};

//This code does not work
For( i = 1, i <= N Items(cols), i++,
       dt << New Column( 
              "2 * " || Char( cols[i] ),
              Numeric,
              "Continuous",
              Formula( As Column( cols[i] ) * 2)
       )
);

New columns are created with the correct name but this formula:

Solution

There are two general strategies to solve this: building the expression while evaluating part of it before the rest, and building the expression as a string and then evaluating it. Note that the variable being inserted can actually be an expression of any length.

Expressions

These methods involve writing an expression and evaluating one part of it before the rest. It can be tricky to find the right combination of Eval, Eval Expr, and Expr, but you can see syntax highlighting and use the script editor to make sure parenthesis are matched.

Eval, Eval Expr

Write the expression and wrap any parts that should be evaluated immediately with Expr():

//Eval Expr
For( i = 1, i <= N Items(cols), i++,
       Eval( Eval Expr(
              dt << New Column( 
                    "2 * " || Char( cols[i] ),
                    Numeric,
                    "Continuous",
                    Formula( As Column( Expr( Char( cols[i] ) ) ) * 2)
              )
       ) )
);

Substitute

Write the expression and replace part of it with evaluated code:

//Substitute
For( i = 1, i <= N Items(cols), i++,
	Eval( Substitute(
		Expr( dt << New Column( 
			"2 * " || cn,
			Numeric,
			"Continuous",
			Formula( As Column( cn ) * 2)
		) ),
		Expr( cn ),
		Char( cols[i] )
	) )
);

String Methods

These methods involve writing the expression as a string, using Parse() to convert it into an expression and then Eval() to actually evaluate it. Downsides of these methods include not seeing any syntax highlighting, needing to handle double-quotes using either \!" or \[ ... ]\, and being generally hard to troubleshoot or modify in the future, especially for long expressions.

Eval Insert

Write the expression as a string with 'in-line' code that is evaluated and placed inside the string, then parse and evalute it:

//Eval Insert
For( i = 1, i <= N Items(cols), i++,
       Eval( Parse( Eval Insert(
              "\[dt << New Column( 
                    "2 * ^Char( cols[i] )^",
                    Numeric,
                    "Continuous",
                    Formula( As Column( "^Char( cols[i] )^" ) * 2)
              )]\"
       ) ) );
);

Substitute

Write the expression in a string with a placeholder that is replaced with the evaluated expression. Note this is the same function as above:

//Substitute
For( i = 1, i <= N Items(cols), i++,
       Eval( Parse( Substitute(
              "\[dt << New Column( 
                    "2 * ColumnName",
                    Numeric,
                    "Continuous",
                    Formula( As Column( "ColumnName" ) * 2)
              )]\",
              "ColumnName",
              Char( cols[i] )
       ) ) );
);

Concatenate

The most basic form, write the expression using '||'.  Note you could use use \[ ... ]\ instead of \!":

//Concatenate
For( i = 1, i <= N Items(cols), i++,
	Eval( Parse(
		"dt << New Column( 
			\!"2 * " || Char( cols[i] ) || "\!",
			Numeric,
			\!"Continuous\!",
			Formula( As Column( \!"" || Char( cols[i] ) || "\!" ) * 2)
		)"
	) );
);

See Also

The 'Expression Handling Functions', section of the Scripting Guide includes these and other methods to accomplish the same thing.  This blog post by @joseph_morgan has addition examples and insights into the same topic.

Thank you @Justin_Chilton for feedback on this post.

[David_Burnham]

Excellent review of the methods available for dynamically constructing JSL expressions.  Master this and everything is fairly straightforward.

[robot]

Very useful!

[Greenhorn]

Creating a table with formula in the columns - Random Normal() - from external jmp 

I've already got a working solution for my proble. Link above.

However, I'm obsesive to get my code working properly.

And this thread seems to respond to my topic.

I tried some of above techniques. But it doesn't work or I don't understand it.

Do you have a suggestion for me? How to get expression "As Column( mydt, :Mean )[1]" resolved to source values?

//not working
Clear Globals();
Names Default To Here( 1 ); // 

mydt = Current Data Table();

// New data table
my_newdt = New Table( "Table Name", 
//	Add Rows( N Rows( mydt ) ), // row count can be set by hand e.g. 1000
	Add Rows( 10 ), // row count can be set by hand e.g. 1000

	New Column( Column( mydt, 1 )[1], // Loop i==1
		Numeric,
		"Continuous",
		Format( "Best", 6 ),
		Formula( Random Normal( As Column( mydt, :Mean )[1], 0.01))// As Column(mydt, :Mean )[1] is evalueted in the target table. But Formula has the expression "As Column( mydt, :Mean )[1]" instead of a value from source table
	)

);// End New data table

/*// For loop. Add Columns
For( i = 2, i <= N Rows( mydt ), i++, // 
	my_newdt << New Column( Column( mydt, 1 )[i], // Loop i ==2
		Numeric,
		"Continuous",
		Format( "Best", 6 ),
		Set Each Value( Random Normal( As Column( mydt, :Mean )[i], As Column( mydt, :Std Dev )[i] ) )//
	)	
	
)// End for loop;
;*/

[ih]

Hi @Greenhorn,

You need to tell JMP to evaluate that expression before putting it into the formula, as once in a column formula it doesn't know how to reference your other table any more. Using my preferred method (the top one above), might look something like this:

Names Default To Here( 1 ); // 

//Table with columns you want to create
mydt = New Table( "Source",
	Add Rows( 3 ),
	New Column( "Name", Character, "Nominal", Set Values( {"a", "b", "c"} ) ),
	New Column( "Mean", Numeric, "Continuous", Format( "Best", 12 ),
		Set Values( [1, 2, 3] ) ),
	New Column( "Std Dev", Numeric, "Continuous", Format( "Best", 12 ),
		Set Values( [0.5, 0.6, 0.7] ) )
);

my_newdt = New Table( "Target", Add Rows( 100 ) );

// For loop. Add Columns
For( i = 2, i <= N Rows( mydt ), i++, // 
	Eval( Eval Expr(
		my_newdt << New Column( Column( mydt, 1 )[i],
			Numeric,
			"Continuous",
			Format( "Best", 6 ),
			Formula( Random Normal( Expr( As Column( mydt, :Mean )[i] ), Expr( As Column( mydt, :Std Dev )[i] ) ) )//
		)
	) )	
);

[Greenhorn]

hi ih,

it worked for  columns b and c. "a" doesn't apear. An empty Column 1 apears instead.

That is why I tried to put the "for loop" inside the New Table (). But then with this approach, I get just the expression inside the formula.

Do you know how to handle "for loop" with formula inside the New Table() in order to get all required columns at once?

[ih]

@Greenhorn ,

The for loop starts with i=2, that is why it skipped the first column, and a single blank column called Column 1 was created when the data table was created. You can 'fix' this by starting the loop with i=1 and deleting Column 1 after, like this:

View more...

Names Default To Here( 1 );

//Table with columns you want to create
mydt = New Table( "Source",
	Add Rows( 3 ),
	New Column( "Name", Character, "Nominal", Set Values( {"a", "b", "c"} ) ),
	New Column( "Mean", Numeric, "Continuous", Format( "Best", 12 ),
		Set Values( [1, 2, 3] ) ),
	New Column( "Std Dev", Numeric, "Continuous", Format( "Best", 12 ),
		Set Values( [0.5, 0.6, 0.7] ) )
);

my_newdt = New Table( "Target", Add Rows( 100 ) );

// For loop. Add Columns
For( i = 1, i <= N Rows( mydt ), i++, // 
	Eval( Eval Expr(
		my_newdt << New Column( Column( mydt, 1 )[i],
			Numeric,
			"Continuous",
			Format( "Best", 6 ),
			Formula( Random Normal( Expr( As Column( mydt, :Mean )[i] ), Expr( As Column( mydt, :Std Dev )[i] ) ) )//
		)
	) )	
);

//delete the first column created with the data table
my_newdt << Delete Columns( :Column 1 );

[Greenhorn]

Thanks a lot! Yes. You are right with the loop start. I didn't realize that.

[FN]

Is there a way where the As Column() is not needed?

Here we need to include it as we are passing a string, I wonder how to do it so the formula is left with :column.

[jthi]

This might have some unnecessary functions there, but you can use Name Expr with column reference (As Column(name)):

Names Default To Here(1);

//Open A Table and select a few columns
dt = Open("$Sample_data/iris.jmp");
cols = {Sepal length, Petal length};

For(i = 1, i <= N Items(cols), i++,
	Eval(Eval Expr(dt << New Column("2 * " || Char(cols[i]), Numeric, "Continuous", Formula(Expr(Name Expr(As Column(Char(cols[i])))) * 2))))
);

[ih]

Here is an example for an expression method and a string method:

Names default to here(1);

//Open A Table and select a few columns
dt = Open( "$Sample_data/iris.jmp" );
cols = {Sepal length, Petal length};

//Eval Expr
For( i = 1, i <= N Items(cols), i++,
	Eval( Eval Expr(
		  dt << New Column( 
				"Eval Expr 2 * " || Char( cols[i] ),
				Numeric,
				"Continuous",
				Formula( Expr( cols[i]  ) * 2 )
		  )
	) )
);

//Substitute with String
For( i = 1, i <= N Items(cols), i++,
       Eval( Parse( Substitute(
              "\[dt << New Column( 
                    "Substitute String 2 * ColumnName",
                    Numeric,
                    "Continuous",
                    Formula( :"ColumnName"n * 2)
              )]\",
              "ColumnName",
              Char( cols[i] )
       ) ) );
);