🙏 Expression Indexing: read and write access

[hogi]

☑ cool new feature
☐ could help many users!

☐ removes a „bug“

☐ nice to have

☐ nobody needs it

What inspired this wish list request? 

edit: first part shifted to Expression Indexing: How to climb the Expression Tree? 

Thanks @jthi for providing a solution:

Via  Extract expr a user has the possibility to search and extract parts of Expressions.
The function provides a pattern search and returns the first match.

Even Wildcards are possible

What is the improvement you would like to see? 

A mixture of Display Tree Indexing and Data Table Indexing - just for expressions:

A more precise read access via:

dt = Open( "$SAMPLE_DATA/Big Class.jmp" );
myExpr=Expr(gb = dt << Graph Builder(
	Size( 437, 413 ),
	Graph Spacing( 4 ),
	Variables( X( :height ), Y( :weight ), Y(:height), Overlay( :sex ) ),
	Elements( Points( X, Y, Legend( 1 ) ), Smoother( X, Y, Legend( 2 ) ) )
));
//instead of:
Arg(Arg(Arg(Arg(Arg(myExpr,2),2),3),3),1)

// I want to use:
myExpr[2][2]["Variables"]["Y"(2)][1]
//assign // send // Graph Builder // Variables

// or like it's possible for Display Trees:
myExpr["Variables"]["Y"(2)][1]

Where it gets really cool: write access similar to what's is possible via Data Table Indexing:

myExpr["Variables"]["Y"(2)][1] = Expr(:height)

Like Substitute - but in a different dimension!

Why is this idea important? 

With the Power of Expression Indexing (read & write (!!) access), Expression Handling in Jmp will get easier

more wishes submitted by  

[jthi]

JMP does have Extract Expr which makes some of this easier

Names Default To Here(1);

myExpr = Expr(
	gb = dt << Graph Builder(
		Size(437, 413),
		Graph Spacing(4),
		Variables(X(:height), Y(:weight), Overlay(:sex)),
		Elements(Points(X, Y, Legend(1)), Smoother(X, Y, Legend(2)))
	)
);

yexpr = Extract Expr(myExpr, Y(Wild()));
variables_expr = Extract Expr(myExpr, Variables(Wild List()));

Names Default To Here(1);

myExpr = Expr(
	gb = dt << Graph Builder(
		Size(437, 413),
		Graph Spacing(4),
		Variables(X(:height), Y(:weight), Y(:height), Overlay(:sex)),
		Elements(Points(X, Y, Legend(1)), Smoother(X, Y, Legend(2)))
	)
);

variables_expr = Extract Expr(myExpr, Variables(Wild List()));
idx = 0;
For(i = 1, i <= N Arg(variables_expr), i++,
	firstlevel = Arg(variables_expr, i); // could use Extract Expr
	If(Head(firstlevel) == Expr(Y) & idx == 2,
		secondlevel = Arg(firstlevel, 1);
	);
);
show(secondlevel);

[hogi]

Thanks @jthi !
Great - Extract expr -  much better than my workaround via Substitute (... List):

-> Fixes the issues 1 & 3 

For issue #2, the second example is a nice workaround - and at the same time:

illustrates the simplicity of Expression Indexing.

With Extract expr - available, there is no urgent need for Expression Indexing

[hogi]

@jthi , with the nice solution, I think the topic has a much better place in the Discussions:
https://community.jmp.com/t5/Discussions/Expression-Indexing-How-to-climbing-the-Expression-Tree/m-p... 

could you please add your comment there - then I will accept is as a solution

[Craige_Hales]

How would you convert this

myExpr = Expr(
	gb = dt << Graph Builder(
		Size(437, 413),
		Graph Spacing(4),
		Variables(X(:height), Y(:weight), Overlay(:sex)),
		Elements(Points(X, Y, Legend(1)), Smoother(X, Y, Legend(2)))
	)
);

to this?

myExpr = Expr(
	gb = dt << Graph Builder(
		Size(437, 413),
		Graph Spacing(4),
		Variables(X(:height), Y(:weight), /*added:*/ Y(:height), Overlay(:sex)),
		Elements(Points(X, Y, Legend(1)), Smoother(X, Y, Legend(2)))
	)
);

Indexing could make it a lot easier. It feels like an InsertInto( myExpr, ... ) problem. That will work if you can figure out how to turn the operators into lists and then back again, but something like

InsertInto( myExpr[/* = */ 2][ /* << */ 2]["Variables"], /* insert position*/ 3, expr( Y(:height))) 

could edit the expression in place.

Proof of concept:

myExpr = Expr(
	gb = dt << Graph Builder(
		Size( 437, 413 ),
		Graph Spacing( 4 ),
		Variables( X( :height ), Y( :weight ), Overlay( :sex ) ),
		Elements( Points( X, Y, Legend( 1 ) ), Smoother( X, Y, Legend( 2 ) ) )
	)
);
Show( "Original", Name Expr( myExpr ) );
// convert operators to lists
Substitute Into( myExpr, Expr( Assign() ), {} );
Substitute Into( myExpr[2], Expr( Send() ), {} );
Substitute Into( myExpr[2][2], Expr( Graph Builder() ), {} );
Substitute Into( myExpr[2][2][3], Expr( Variables() ), {} );
// make the edit
Insert Into( myExpr[2][2][3], Expr( Y( :height ) ), 3 );
// reverse the earlier conversions
Substitute Into( myExpr[2][2][3], {}, Expr( Variables() ) );
Substitute Into( myExpr[2][2], {}, Expr( Graph Builder() ) );
Substitute Into( myExpr[2], {}, Expr( Send() ) );
Substitute Into( myExpr, {}, Expr( Assign() ) );
Show( "Edited", Name Expr( myExpr ) );

//
// wouldn't it be nice if we could use the edit line directly?
//
Write( "\!nTry without a list:" );
Insert Into( myExpr[2][2][3], Expr( Y( :height ) ), 3 ); // does not work
// it fails because expressions can't be indexed, even though they
// are extremely similar to lists (as seen above).

// I'm not sure why it needs to be that way.

We like using the xxxInto(...) functions because they modify an existing expression tree, in place. In some cases that could be an efficiency concern (vs making copies of fragments of an expression and rebuilding) but it is also a bit more clear to treat the operators as lists.

Thanks @hogi , @jthi 

[hogi]

Thanks @Craige_Hales  for the nice example

I hope, others will support the idea as well ...

[hogi]

JMP Expr() weirdness, can anyone explain? 

For Insert Into, besides the current turbo mode, please provide an additional  "non-merge" mode ....

turbo mode (as Insert Into behaves now):

if the Head of the expression to be inserted is the same as the Head of the level above the target position, the Expression is "merged" with the higher level.

Insert Into (List(a,b), List(c,d)) // {a,b,c,d}

additional non-merge mode:

Insert Into (List(a,b), List(c,d), merge(0)) // {a,b,{c,d}}