Script question for Rerun Formulas

[#1]

I asked this question on a prior post but that was a few weeks ago so it may not get noticed.  I suspect this is an easy scripting question for this group.  Below is a script (donated by frank) that successfully runs a Monte Carlo simulation for a Stepwise Regression.  The only thing it did not do was to recalculate a number of columns where I have random numbers in formulas before conducting the Stepwise regression.  The <<Rerun Formulas command works when I manually click on the red arrow in the table and say Rerun Formulas, but in the script it doesn't do anything.  Can someone help me get the formulas to Rerun before each iteration in the For loop?

My unsuccessful attempt is line 19 in the script.  Thanks.

Clear Globals();
Clear Log();
Names Default To Here( 1 );

//Define path in and out
Path_in = Pick File( "Select file" );

If( Directory Exists( "$Desktop\Fit_Stepwise" ),
	,
	Create Directory( "$Desktop\Fit_Stepwise" )
);

Path_out = "$Desktop\Fit_Stepwise\";

//Run Fit model and stepwise after open datatable
dt = Open( Path_in );

For( i = 1, i <= 1000, i++,
	dt << Rerun Formulas;
	ow = dt << Fit Model(
		Y( :Y ),
		Effects(
			:Column 2 001,
			:Column 2 002,
			:Column 2 003,
			:Column 2 004,
			:Column 2 005,
			:Column 2 006,
			:Column 2 007,
			:Column 2 008,
			:Column 2 009,
			:Column 2 010
		),
		Personality( "Stepwise" ),
		Run,
		SendToReport( Dispatch( {}, "Step History", OutlineBox, {Close( 1 )} ) )
	);

	dt_result = Report( ow )[Outline Box( "Stepwise Fit for Y" )][Outline Box( "Current Estimates" )][Table Box( 1 )] << Make Into Data Table;
	dt_result << save( Path_out || Char( i ) || ".csv", "csv" );
	ow << close window;
	dt_result << Close Window;
);

dt << Close Window;

dt_result = Multiple File Import(
	<<Set Folder( Path_out ),
	<<Set Name Filter( "*.*;" ),
	<<Set Name Enable( 0 ),
	<<Set Size Filter( {589, 589} ),
	<<Set Size Enable( 0 ),
	<<Set Date Filter( {3727681987.773, 3727682566.676} ),
	<<Set Add File Name Column( 1 ),
	<<Set Import Mode( "CSVData" ),
	<<Set Charset( "Best Guess" ),
	<<Set Stack Mode( "Stack Similar" ),
	<<Set CSV Has Headers( 1 ),
	<<Set CSV Allow Numeric( 1 ),
	<<Set CSV First Header Line( 1 ),
	<<Set CSV Number Of Header Lines( 1 ),
	<<Set CSV First Data Line( 2 ),
	<<Set CSV EOF Comma( 1 ),
	<<Set CSV EOF Tab( 0 ),
	<<Set CSV EOF Space( 0 ),
	<<Set CSV EOF Spaces( 0 ),
	<<Set CSV EOF Other( "" ),
	<<Set CSV EOL CRLF( 1 ),
	<<Set CSV EOL CR( 1 ),
	<<Set CSV EOL LF( 1 ),
	<<Set CSV EOL Semicolon( 0 ),
	<<Set CSV EOL Other( "" ),
	<<Set CSV Quote( "\!"" ),
	<<Set CSV Escape( "" ),
	<<Set Import Callback( Empty() )
) << Import Data;

dt_result << Row Selection( Select where( :Name( "\!"Prob>F\!"" ) > 0.05 ) ) << Delete Rows;
dt_result << Distribution( Stack( 1 ), Nominal Distribution( Column( :Parameter ), Horizontal Layout( 1 ), Vertical( 0 ) ) );

[#2]

Rerun formulas should work, maybe you need to add wait(0) somewhere? Without seeing the datatable it is a bit difficult to test where the issue could be in (row count, column count, complicated formulas, random seed not changing for some reason).

With this simple example the values seem to update fine:

Names Default To Here(1);

dt = New Table("Untitled",
	Add Rows(5),
	New Column("Column1", Numeric, "Continuous", Format("Best", 12), Formula(Random Normal(1, 1))),
	New Column("Column2", Numeric, "Continuous", Format("Best", 12), Formula(Random Normal(2, 1)))

);

For(i = 1, i <= 3, i++,
	dt << Rerun Formulas;
	Show(dt:Column1 << get as matrix);
	Show(dt:Column2 << get as matrix);
	fm = dt << Fit Model(
		Y(:Column1),
		Effects(:Column2),
		Personality("Stepwise"),
		Run,
		SendToReport(Dispatch({}, "Step History", OutlineBox, {Close(1)}))
	);

	dt_result = Report(fm)[Outline Box("Stepwise Fit for Column1")][Outline Box("Current Estimates")][Table Box(1)] << Make Into Data Table;
	fm << close window;
	//dt_result << Close Window;	
);

[#3]

I've attached the file.  It looks like I inserted Rerun Formulas in the correct place but it isn't working.  How and where does Wait go?

[#4]

Just to make sure that column will be evaluated, add this before For loop (if you are using earlier version than JMP16 change For Each to For):

For Each({col_name}, dt << get column names(String),
	Column(dt, col_name) << suppress eval(False)
);

In the table I opened the formula columns had evaluation suppressed, but it might just be safety feature with downloaded files.

If you have to add wait(0), you can go a bit excessive with to start with and almost add it after every row inside the for-loop. Usually there is no need for it in JMP, but there are some cases where it is required.

[#5]

The formulas in my table are not suppressed.  Can you explain a bit more about the commands you are suggesting I add?  I am almost completely ignorant about scripting (and don't really want to spend my time becoming more proficient - don't spend your energy telling me why I should - I am a special case in many ways).  Do I put one column name inside the { } or put all 6 columns (separated by , or ;?)?  Do I need to replace String with anything?  And, do I list Column separately for the suppress eval (false) command?

Regarding wait() I tried that unsuccessfully, but I can't find any documentation in the scripting index on the wait() command.

[#6 · Accepted solution]

I'm not sure why the original script you had wouldn't update the values in the formula columns. You could maybe test this further by lowering the loop-count (use for example as max value) and adding Wait(1); for example after the <<rerun formulas, this should allow you to see changes in your datatable.

Here is slightly modified script using the subset table you provided earlier. As you don't want to be more proficient with scripting, I won't go very deep on explaining how the functions here work, but see explanation below and comments in the script, I can also explain more if needed. The script provided here should:

1. Loop over all columns found in dt and enable evaluation on them to allow recalculation on formula columns
2. Run Fit Model 100 times and recalculate values in formula columns each time with <<Rerun Formulas
   1. Will also use Show() to print Col Median() for one column to log. This will slow down the script execution, but should show that the values in column do change
3. Create one result table with the results and close all unnecessary tables and windows
4. After running loop has finished: hide and exclude rows with ProbF > 0.05 and then create distribution

Names Default To Here(1);

dt = Current Data Table();
//dt = Open("Subset of Test data.jmp");
dt << Show Window(0); // should provide slight speed improvement

For Each({col_name}, dt << get column names(String), Column(dt, col_name) << suppress eval(False));

get_stepwise = Expr(
	dt << Rerun Formulas;
	Show(Col Mean(As Column(dt, :Column 2 001))); //will slow down script execution, but shows that values in Column 2 001 should change
	Show(Col Mean(As Column(dt, :Column 2 010))); 
	fm = dt << Fit Model(
		Y(:Y),
		Effects(
			:Column 2 001, :Column 2 002, :Column 2 003, :Column 2 004, :Column 2 005, :Column 2 006, :Column 2 007, :Column 2 008, :Column 2 009,
			:Column 2 010
		),
		Personality("Stepwise"),
		Run,
		SendToReport(Dispatch({}, "Step History", OutlineBox, {Close(1)})),
		Invisible // should provide slight speed improvement
	);

	dt_result = Report(fm)[Outline Box("Stepwise Fit for Y")][Outline Box("Current Estimates")][Table Box(1)] << Make Into Data Table(invisible);
	dt_result << New Column("Loop", Numeric, Continuous, <<Set Each Value(i)); // to separate different "result tables"
	fm << close window;
);

// run first to get "collection datatable"
i = 1;
get_stepwise;
dt_collector = dt_result;

// run loop from starting 2, because we did already run i = 1 
For(i = 2, i <= 100, i++,
	get_stepwise;
	dt_collector << Concatenate(dt_result, Append to first table); // concatenate to collect results
	Close(dt_result, no save);
);
Close(dt, no save); // close original table
dt_collector << Show Window(1); // show collector table

// hide and exclude rows with prob over 0.05
dt_collector << Select where(:"\!"Prob>F\!""n > 0.05) << Hide And Exclude(1) << Clear Select;

// plot distribution
dt_collector << Distribution(Stack(1), Nominal Distribution(Column(:Parameter), Horizontal Layout(1), Vertical(0)));

This script didn't seem to have no need for Wait(0) at least on my setup (fast PC, Windows 10 and JMP16.2). For me it seems like the formula values did get re-evaluated

Results:

Scripting index for Wait(), clicking Topic Help sometimes provides more info (in case of Wait() it does)

JMP Help - Wait(n) "Specifying Wait(0) enables one cycle of message processing."

[#7]

Thank you, this is perfect.  And thank you for being patient with me.  Also, the help on Wait() is in help but not the scripting index, as you say.