Scripting automation: Retrieving predictors & setting sigma level

[#1]

Hello,

I was given a project to script a modeling flow. All predictors and responses are numeric continuous & responses are fit separately. After some initial data prep...

1. Stepwise K-fold validation <-- Automated
2. Step 1's model run in SLS <-- Automated
3. SLS prediction profiler:
   1. Initialize Simulator <-- Automated
   2. Change all factor types to random <-- Not automated
      
      1. Change all SD's to 3 sigma value for each column <-- Not automated 

This flow will use a for loop. During each iteration it will do a battery of analysis to a single response variable. The script for 3.2 is easy to get, but changing the SD value to the 3 sigma value isn't. How should the automation of modifying the SD value for 3.2 work while using a for loop? Is there a better way? I've looked into preferences > platforms > fit model > fir least squares and I don't see an option to change the SD value from 1 SD to 3 SD.

Newly updated to JMP 17, if that helps any.

[#2]

Something like this?

Names Default To Here(1);
dt = Open("$SAMPLE_DATA/Tiretread.jmp");
obj = Profiler(
	Y(
		:Pred Formula ABRASION, :Pred Formula MODULUS, :Pred Formula ELONG,
		:Pred Formula HARDNESS
	)
);

val = 3;

obj << Simulator(
	1,
	Factors(
		SILICA << Random(Normal(1.25, 0.3266)), SILANE << Random(Normal weighted(50, 6.532)),
		SULFUR << Fixed(2.25)
	),
	Responses(
		Pred Formula ABRASION << Add Random Noise(1), Pred Formula MODULUS << Add Random Noise(2),
		Pred Formula ELONG << Add Random Noise(val), Pred Formula HARDNESS << Add Random Noise(val*2)
	)
);

Modified from example in Scripting Index

[#3]

Hello @jthi ,

Not quite. Below is a snippet of code of what I have, I'll explain more in a bit.

// Iterating through responses  - JMP 17
	mymodel_sw = Fit Model( Y( response[i] ),
		Effects( Factorial to degree( eval( predict ) ), Response Surface( eval( predict ) ) ),
		Personality("Stepwise"),
		Run( "K-Fold Crossvalidation"n(5) )	
	);
	
	// Runs modeling
	mymodel_sw << Go;
	
	// Finishes predictor selection before proceeding
	mymodel_sw << Finish;
	
	// Makes model
	mymodel_sls = mymodel_sw << Make Model;
	
	// Runs SLS model
	mymodel_sls << Run;
	
	// Finishes modeling before proceeding
	mymodel_sls << Finish;
	
	// Gets list of predictors
	param_SLS = mymodel_sls << Get Parameter Names;

Once I << Get Parameter Names, param_SLS errors with Deleted Object Reference.  However, I need this list of parameters to build out the arguments in << Simulate to table, which I'll do by creating a custom function that takes care of that.

This snippet of code is within a for loop that is iterating through a list of responses, one at a time.  Does this make sense?

[#4]

Or is there a way to retrieve the parameter names from the profiler tool, so I don't get all of the interaction terms and have to sort through all of that? I'd like a unique list of parameter names I can work with. The number of parameters that will be fit I must assume will differ per model. 

[#5]

Here's a much more in-depth sample script.  Note that this does NOT work!

Names default to here(1);

dt = open( "$SAMPLE_DATA/Body Fat.jmp" );

/////////////////////////// Custom function - Help!

mysim = function( {list, iteration, stddev, step},
	
	// New list placeholder
	unique_factors = {};
	
	// Adds only unique factors to new list, uses "*" as identifier
	for( i = 1, i <= N items( list ), i++,
		if( list[i] << !contains( "*" ) , insert into( unique_factors, list[i] )		
		)		
	);
	
	num_sim_factors = N Items( unique_factors );
	
	// Modifies each factor in the simulator to normal distribution with 3 standard deviations
	simfactor = {};
	
	for( i = 1, i <= num_sim_factors, i++,
		temp_1 = char( unique_factors[i] ) || " << Random( Normal( " ||  char( col mean( unique_factors[i] ) ) ||
					", " || char( stddev * col std dev( unique_factors[i] ) ) || ") ),"
		insert into( sim_factor, temp_1 )
	);
	
	// Inputs simfactor into simulator
	mymodel_sls << Simulator( 1, eval( simfactor ) );
	
	// Creates the arguments for Simulate to table
	simtable_pt1 = "N Runs( " || char( iteration ) || "),"
	simtable_pt2 = {};
	
		// Creates an argument for each factor
		for( i = 1, i <= num_sim_factors, i++,
			temp_2 = char( unique_factor[i] ) || " << Sequence Location( " || char ( Col Minimum( unique_factor[i] ) ) || ", "
				chart ( Col Maximum( unique_factor[i] ) ) || ", " || char( step ) || " ),";
			insert into( simtable_pt2, temp_2 )
		);
	
	// Combines strings into a single argument for use for Simulate to Table
	simarg = simtable_pt1 || simtable_pt2;
	
	// Simuates data to table for further analysis
	mymodel_sls << Get simulator;
	mymodel_sls << Simulate to table( simarg );
);

/////////////////////////// Cols to use - user adjustable

predict = {:"Neck circumference (cm)"n, :"Chest circumference (cm)"n,
	:"Abdomen circumference (cm)"n, :"Hip circumference (cm)"n,
	:"Thigh circumference (cm)"n, :"Knee circumference (cm)"n,
	:"Ankle circumference (cm)"n, :"Biceps (extended) circumference (cm)"n,
	:"Forearm circumference (cm)"n, :"Wrist circumference (cm)"n};

response = {:"Age (years)"n, :"Weight (lbs)"n, :"Height (inches)"n};

///////////////////////////  User input

num_response = N Items( response );
num_predict = N Items( predict );

// How many simulations to run?
sim_no = 1000;

// How many standard deviations for each predictor in the simulation?
stddev = 3;

// Steps in simulation
steps = 4;

///////////////////////////  Changing col attribute

// In a later step, Simulate, the mean should be 0 due to this
for( i = 1, i <= num_response, i++,
	response[i] << Set Property( "Coding", { col minimum( response[i] ), col maximum( response[i] ) } );
);

///////////////////////////  Modeling

/* Fit Model - Stepwise, w/ K-fold validation --> SLS --> Simulate */
for( i = 1, i <= num_response, i++,
	
	// Names models for calling later
	mymodel_sw = "FM_SW_" || char( i );

	// Iterating through responses  - JMP 17
	mymodel_sw = Fit Model( Y( response[i] ),
		Effects( Factorial to degree( eval( predict ) ), Response Surface( eval( predict ) ) ),
		Personality("Stepwise"),
		Run( "K-Fold Crossvalidation"n(5) )	
	);
	
	// Runs modeling
	mymodel_sw << Go;
	
	// Finishes predictor selection before proceeding
	mymodel_sw << Finish;
	
	// Makes model
	mymodel_sls = mymodel_sw << Make Model;
	
	// Runs SLS model
	mymodel_sls << Run;
	
	// Finishes modeling before proceeding
	mymodel_sls << Finish;
	
	////////////////// NOT WORKING BELOW
	
	// Gets list of predictors <--- This is first step
	param_sls1 = mymodel_sls << Get Effect Names;
	param_sls2 = mymodel_sls << Get Parameter Names;
	show(param_sls1, param_sls2);
	
	// Calls on custom function <--- This is second step
	mysim( param_sls1, sim_no, stddev, steps );
	
	// Names Simulate to table dataset by response variable in for loop
	Current data table() << Set name( char( response[i] ) || "_report" );	
	
);

Help! There are multiple parts of this not working!  Here is what's working - Comment out the custom function and anything below (near the bottom of the script) where is states NOT WORKING BELOW. I can get through the generation of the SLS modeling, but once it comes to modifying the simulate parameters and running the simulate, both of which are customized (see custom function), I'm out of ideas.

[#6]

There are lots of typos on that function which can cause issues. I just fixed some things, no idea if it does what it is supposed to do

Names Default To Here(1);

dt = Open("$SAMPLE_DATA/Body Fat.jmp");

mysim = function({input_list, iteration, stddev, step},

	unique_factors = Filter Each({item}, input_list, !Contains(item, "*"));
	
	num_sim_factors = N Items(unique_factors);
	simfactor = {};
	
	simfactor = Transform Each({uniq_factor}, unique_factors,
		Eval Insert("^uniq_factor^ << Random(Normal(^col mean(column(dt, uniq_factor))^, ^stddev * col std dev(column(dt, uniq_factor))^");
	);
	
	// Inputs simfactor into simulator
	mymodel_sls << Simulator( 1, eval(simfactor));
	
	// Creates the arguments for Simulate to table
	simtable_pt1 = "N Runs( " || char(iteration) || "),";
	simtable_pt2 = {};
	
	// Creates an argument for each factor
	simtable_pt2 = Transform Each({uniq_factor}, unique_factors,
		Eval Insert("^uniq_factor^ << Sequence Location(^Col Minimum(column(dt, uniq_factor))^, ^Col Maximum(column(dt, uniq_factor))^, ^step^)");
	);
	
	// Combines strings into a single argument for use for Simulate to Table
	simarg = simtable_pt1 || Concat Items(simtable_pt2, ",");
	
	// Simuates data to table for further analysis
	mymodel_sls << Get simulator;
	mymodel_sls << Simulate to table(simarg);
);

predict = {:"Neck circumference (cm)"n, :"Chest circumference (cm)"n,
	:"Abdomen circumference (cm)"n, :"Hip circumference (cm)"n,
	:"Thigh circumference (cm)"n, :"Knee circumference (cm)"n,
	:"Ankle circumference (cm)"n, :"Biceps (extended) circumference (cm)"n,
	:"Forearm circumference (cm)"n, :"Wrist circumference (cm)"n};

response = {:"Age (years)"n, :"Weight (lbs)"n, :"Height (inches)"n};

num_response = N Items(response);
num_predict = N Items(predict);

sim_no = 1000;
stddev = 3;
steps = 4;

For(i = 1, i <= num_response, i++,
	response[i] << Set Property("Coding", {Col Minimum(response[i]), Col Maximum(response[i])})
);

For(i = 1, i <= num_response, i++, 
	mymodel_sw = "FM_SW_" || Char(i);

	mymodel_sw = Fit Model(
		Y(response[i]),
		Effects(Factorial to degree(Eval(predict)), Response Surface(Eval(predict))),
		Personality("Stepwise"),
		Run("K-Fold Crossvalidation"n(5))
	);
	
	mymodel_sw << Go;
	mymodel_sw << Finish;
	
	mymodel_sls = mymodel_sw << Make Model;
	mymodel_sls = mymodel_sls << Run;
	mymodel_sls << Finish;
	
	
	param_sls1 = mymodel_sls << Get Effect Names;
	param_sls2 = mymodel_sls << Get Parameter Names;
	Show(param_sls1, param_sls2);
	
	mysim(param_sls1, sim_no, stddev, steps);
	
	dt << Set name(Char(response[i]) || "_report");
);

[#7]

@jthi - I'm going to have to dig into your recommendation for a bit.  I've seen Filter Each, Transform Each and Eval Insert, but I have no experience with them and do not yet know how they work.

Another question for you.  In Eval Insert, you have carrot ^ marks that I've never seen before.  Is there some documentation I can learn more about their functionality?  I think it's pretty clear from how you're using it, but I'm intrigued and want to make sure I'm interpreting your usage correctly.

simfactor = Transform Each({uniq_factor}, unique_factors,
		Eval Insert("^uniq_factor^ << Random(Normal(^col mean(column(dt, uniq_factor))^, ^stddev * col std dev(column(dt, uniq_factor))^");
	);

[#8 · Accepted solution]

@jthi ,

You were on the right track! Other expression manipulations were necessary to create the arguments to insert into their respective functions.  In the mysim custom function, the arguments insert into modifying the distribution for the simulation, and generating the argument to insert into Simulate to Table have been explained in detail.  This is a very exciting learning opportunity and opened my eyes to these types of advanced expressions.

If anyone wants to know more about expression manipulations, here's a link to documentation: https://www.jmp.com/support/help/en/17.0/#page/jmp/advanced-expressions-macros-and-lists.shtml#

Thank you to those who helped!

Names default to here(1);

dt = open( "$SAMPLE_DATA/Body Fat.jmp" );

mysim = function( {list, iteration, stddev, step}, 
	
	// filter list of effects down to main effects.  These are the effects whose name matches a column name
	// take advantage of the speed of associative arrays to get the intersection set.
	list << Intersect(Associative Array(dt<<Get Column Names("String")));
	unique_factors = list << Get Keys; // get list of values remaining in list after intersection.
	
	num_sim_factors = N Items( unique_factors );
	
	// get reference to profiler in fit model
	prof = Report(mymodel_sls)["Prediction Profiler"] << Get Scriptable Object;
	
	/***  Build the expression to turn on the Simulator with random normal distributions for factors  ***/
	sim_factor = Expr(Factors()); // Base expression to add arguments to later.
	
	// Loop through list of column names and add distribution specification as arguments to the Factors expression
	for( i = 1, i <= num_sim_factors, i++,
		Insert Into(sim_factor,
			EvalExpr(Expr(AsName(unique_factors[i])) << Random( Normal( 
				Expr(col mean( Column(dt, unique_factors[i]) )), // evaluates to the column mean
				Expr(stddev * col std dev(Column(dt, unique_factors[i] ))) // evaluates to a multiple of the column std dev
			))); // result of EvalExpr() is an expression that gets inserted as an argument into the sim_factor expression
		);
	);
	
	// use the sim_factor expression as an argument to the Simulator message sent to the profiler
	// to turn on the simulator with appropriate distributions for the factors.
	Eval(EvalExpr(prof << Simulator( 1, Expr(NameExpr( sim_factor )) )));
	sim = prof << Get Simulator; // get a reference to the simulator object now that is it turned on
	/************************************************************************************************/
	
	/***  Build the expression to simulate to table with desired ranges, steps, and iterations  ***/
	toTableExpr = Expr(Simulate to Table()); // create empty expression for simulate to table
	InsertInto(toTableExpr, EvalExpr(N Runs(Expr(iteration)))); // add the N Runs argument
	
	// to see the expression at this point, you can use NameExpr(), and it will write it to the log
	NameExpr(toTableExpr);

	// Creates an argument for each factor
	for( i = 1, i <= num_sim_factors, i++,
		Insert Into(toTableExpr, // insert another argument into the expression
			EvalExpr(Expr(AsName(unique_factors[i])) << Sequence Location(
				Expr(Col Minimum(Column(dt, unique_factors[i]))), // evaluates to the col min of the current column
				Expr(Col Maximum(Column(dt, unique_factors[i]))), // evaluates to the col max of the current column
				Expr(step) // evaluates to the value of the variable step
			)) // result of EvalExpr() is an expression with numbers inserted as arguments to Sequence Location()
		);
	);
	// at this point, toTableExpr contains an expression that can be sent to the sim object as a message	
	SimDT = Eval(EvalExpr(sim << Expr(NameExpr(toTableExpr)))); // simulate to table with set sequence and N Runs.
	/********************************************************************************************/
	
	// The Simulate to Table message returns a reference to the simulation table.  Meanwhile, the function returns
	// the result of the last evaluation (without specifying otherwise).  Since that is the table, we can set a
	// variable equal to the function call in the main script and we have a reference to the simulation table
	// created inside this function.
);

/////////////////////////// Cols to use - user adjustable

predict = {:"Neck circumference (cm)"n, :"Chest circumference (cm)"n,
	:"Abdomen circumference (cm)"n, :"Hip circumference (cm)"n,
	:"Thigh circumference (cm)"n, :"Knee circumference (cm)"n,
	:"Ankle circumference (cm)"n, :"Biceps (extended) circumference (cm)"n,
	:"Forearm circumference (cm)"n, :"Wrist circumference (cm)"n};

response = {:"Age (years)"n, :"Weight (lbs)"n, :"Height (inches)"n};

///////////////////////////  User input

num_response = N Items( response );
num_predict = N Items( predict );

// How many simulations to run?
sim_no = 10;

// How many standard deviations for each predictor in the simulation?
stddev = 3;

// Steps in simulation
steps = 2;

///////////////////////////  Changing col attribute

// In a later step, Simulate, the mean should be 0 due to this
for( i = 1, i <= num_response, i++,
	response[i] << Set Property( "Coding", { col minimum( response[i] ), col maximum( response[i] ) } );
);

///////////////////////////  Modeling

/* Fit Model - Stepwise, w/ K-fold validation --> SLS --> Simulate */
for( k = 1, k <= num_response, k++,
	
	// Names models for calling later
	mymodel_sw = "FM_SW_" || char( k );

	// Iterating through responses  - JMP 17
	mymodel_sw = Fit Model( Y( response[k] ),
		Effects( Factorial to degree( eval( predict ) ), Response Surface( eval( predict ) ) ),
		Personality("Stepwise"),
		Run( "K-Fold Crossvalidation"n(5) )	
	);
	
	// Runs modeling
	mymodel_sw << Go;
	
	// Finishes predictor selection before proceeding
	mymodel_sw << Finish;
	
	// Makes model
	mymodel_sls = mymodel_sw << Make Model;
	
	// Runs SLS model
	mymodel_sls = mymodel_sls << Run; // this creates another object.  assign your variable to that object
	mymodel_sls << Profiler(1); // make sure profiler is on.
	
	// Gets associative array of effect names.
	// use this in the function to easily filter down to the main effect names
	param_sls1 = Associative Array(mymodel_sls << Get Effect Names);	
		
	// Calls on custom function <--- This is second step
	resultDT = mysim( param_sls1, sim_no, stddev, steps );  // assign variable to reference returned table
	
	// Names Simulate to table dataset by response variable in for loop
	resultDT << Set name( char( response[k] ) || "_report" );		
);

ss