Adding/removing effects in stepwise regression via JSL and column number/name

[#1]

Hi everyone!

I really need help on this because I'm using JMP and JSL for my thesis. I am current using JMP via JSL to analyze some data, and my focus is stepwise regression. I'll have some code snippets attached by the way for ease in helping.

Basically

1.) How to add multiple effects on the initial model via column number?

Currently, I can set Y using only the column number I want, but I can't add X's using the column number.

I want it to go from this

obj = dt << Fit Model(Y(:Column Name(i)),
Effects(
:Effect1Name, :Effect2Name, /*and so on and so forth*/
),
Personality(Stepwise), Run Model(Stopping Rule(MinimumBIC))
);

to this

obj = dt << Fit Model(Y(:Column Name(i)),
Effects(
:Column(/*indices of columns to place as effects*/)
),
Personality(Stepwise), Run Model(Stopping Rule(MinimumBIC))
);

2.) How to remove effects that are least significant?

After running the original script, I see that some effects are not really that significant. The default scripting will be to use

obj << Remove(:/*hardcoded column name*/);

but I want it to be flexible so I can automate it so that it looks like this

obj << Remove(:Column Name(indextoremove));

but the above snippet produces this error message

Term not found in access or evaluation of 'Column Name' , Column Name/*###*/(indextoremove)

Thanks!

[#2 · Accepted solution]

Hi @pedbundoc,

Welcome to the community and good luck on your thesis. 

Unlike other dialog inputs, the Effects input is not just a list of columns.  You also specify higher order effercts in that argument.  You can build the list of effects using things like "column(1), column(2), column(3), column(1)*column(2), column(2)*column(2)" etc.  Also, you cannot (I believe) specify multiple indices within the Column() function in any scenario.  A better solution would be to get a list of column references and subset the list using your column indices like so:

dt = Current Data Table();
all_cols = dt << Get Column Reference();

//let's say factors are columns 2-4
effect_cols = all_cols[2::4];

Now, JMP doesn't seem to like letting me just insert the variable effect_cols directly as the Effects argument for the Fit Model script.  It looks for a column named "effect_cols" rather than evaluating the list.  Therefore, you parse the Fit Model script from a string where we substitute in the evaluation of effect_cols like so:

Step_Str = "\[step = dt << Fit Model(
	Y( Column(5) ),
	Effects(
		^effect_cols^
	),
	Personality( "Stepwise" ),
	Run
);]\";

eval(parse(eval insert(Step_Str)));

This should work for you if you only have main effects.  Otherwise, you'll have to be a bit more creative.  Still totally do-able.

Now, for the issue of removing effects.  Remove is only going to work if you give it a string argument matching the names of one of the effects.  You can use an eval(parse(eval insert())) sequence like we did before to parse a string where we sub in a variable containing the effect name we want to remove.  Here's an example where I take the effect with the largest p-value and remove it:

//Get a reference to the report window containing the stepwise object
step_rprt = step << Report;

//get the term names and current p-values from Current Estimates
terms = step_rprt["Current Estimates"][StringColBox(1)] << Get;
pvals = step_rprt["Current Estimates"][NumberColBox(5)] << Get;

//Remove Intercept from each
remove from(terms,1); //remove intercept
remove from(pvals,1); //remove p-value for intercept

max_index = Contains(pvals, max(pvals)); //get index of max p-value

//create an expression to remove the term with the largest p-value
remove_worst = parse(eval insert("\[step << Remove("^terms[max_index]^");]\"));

//Evaluate the expression
eval(remove_worst);

Hope that helps!

[#3]

Hi @cwillden,

Thanks for replying! You have greatly resolved my first concern (adding multiple effects)!

On the second concern, I am already able to remove the term with the largest p-value. However, I believe it is not the criteria I am using to remove a term.

What I am doing (if I am doing this on a point-and-click method and not scripting) is:

1. Load the table.
2. Run the Fit Model module
   
   1. Add the Y, and X's.
   2. Select stepwise regression
3. Build the model. (Click on Go)
4. Click on Run to display the Report.
5. Go to the Parameter estimates table.
6. Inspect and remove terms based on the Prob>|t|.

I believe the code you sent was removing terms based on the Prob>F of the Current Estimates table of the Fit Stepwise window. What I want to do is remove terms based on the Prob>|t| of the Parameter Estimates table of the Report: Fit Model window.

Again, thanks!

[#4]

Hi again @cwillden!

I changed something in my code, inspired by what I was previously doing and in conjuction with your code snippet. And now, it works!

Thanks you very much!

[#5]

^effect_cols^ doesnt appear to work within eval(parse(eval insert(Step_Str))); as the brackets "{" of the list still exist.