cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Choose Language Hide Translation Bar

Option or function to evaluate a custom column formula at once, or cache values between rows

Some built in column formulas like Col Mean() are very fast because they only calculate expensive values once and return or modify them for each row.  With custom formulas this is harder to achieve because each row is evaluated separately.  Sometimes using a separate column to store intermediate values can speed things up (for example storing the row indexes that meet a certain criteria, instead of re-calculating them every row), but that gets messy and hard to follow. If there were some concept of a column context such that the evaluation of one row could access intermediate values calculated in the previous row, then a script could calculate results for all rows at once and then quickly return them and pass the intermediate results to the next row. The formula would also need access to the whole column instead of just the current row, so the column reference should be passed unevaluated.

16 Comments
ih
Super User (Alumni)
Super User (Alumni)

This option or function would make it simple to create a custom column transformation to solve the problem described by @hogi in one of the later replies to count unique 

hogi
Level XII

@ih wrote:

The formula would also need access to the whole column instead of just the current row, so the column reference should be passed unevaluated.


@Jeff_Perkinson , as for TS-00063463, I think the sad answer is: not yet.

So, besides speed, it's just that many tasks are just impossible to be implemented via custom functions at the moment.

 

status: Investigating -> sounds promising

I hope there is a way to tune Custom Functions to accept column references as a parameter in one of the next releases ...

https://community.jmp.com/t5/Discussions/Column-reference-in-custom-function/m-p/265085/highlight/tr...

hogi
Level XII

ouch - the answer is even sadder: *)
 

hogi_0-1699302358503.png

 

*) TS-00063463

Jeff_Perkinson
Community Manager

@hogi - You are taking my response in the Tech Support case out of context here. I was referring to Custom Transforms since that's what you asked about.

 

I didn't say anything about custom functions in general.

 

Within the formula editor itself, there are usually ways to pass a column reference to a custom function, for example wrapping it in Expr(). Or ways for JMP to recognize that it should use a the column, for example in message sending.

 

hogi
Level XII

Thank you for the clarification.

 

How can I use a column reference in a custom function via "message sending"?

 

My thoughts behind the last post were along the line:
As soon as one can use column references as arguments in custom functions (A), such custom functions could be used as custom transforms as well (B).

(A  -> B)

So, from ¬B , I concluded that A gets unlikely.

hogi
Level XII

Half of a solution - or let's say 1/4:

 

hogi_0-1699382391321.png

is a quite useful feature which I learned some while ago from a solution provided by @txnelson :
https://community.jmp.com/t5/JMP-Wish-List/Option-or-function-to-evaluate-a-custom-column-formula-at... 

 

If you know that the one-time part will always give the same result, you could consider using As Constant a custom function.

 

(below code adapted from https://community.jmp.com/t5/Discussions/Differentiate-between-entire-column-and-current-row/m-p/677... by @jthi )

The interesting thing: it will just be evaluated once -  and never again (!) 

In general, this will be a too extreme version of "once"

 

Any possibility to change the behavior of As Constant in a Custom Formula ? 

E.g. run it every time row()==1?

 

 

Names Default To Here(1);

row_count = 10;
dt = New Table("test",
	Add Rows(row_count),
	New Column("Die X", set each value(row())),
	New Column("Die Y", set each value(Random Integer(1, row_count * 2)))
);


dt << New Column("Formula As Constant", Numeric, Nominal, Formula(
	As Constant(print("direct","");x_m = :Die X << get values);
	Contains(x_m, :Die Y) > 0;
));


Add Custom Functions(
	New Custom Function(
		"test",
		"myFunc",
		Function( {},
			As Constant(print("custom","");x_m2 = :Die X << get values);
			Contains(x_m2, :Die Y) > 0;
)));

dt << New Column("Formula via Custom Formula", Numeric, Nominal, Formula(test:myFunc()));