## Iteration Questions: Data Filtering, and File Saving-Naming

Community Trekker

Joined:

Feb 10, 2015

Hello,

I am wondering how to setup a code that iteratively runs an analysis according to a specific data filter, and then repeat that analysis for a new data filter. The data filter is set to exclude rows based on the presence of a 1 or 0 in a column, and the columns are in sequential order. Here is an example script:

dt = Open("Hyperspectral_DataCleanedOrdered.jmp");

dt << Data Filter(

Mode(Include(1)),

Inverse,

Columns(Rand001),

Where(:Rand001 >= 0.875 & :Rand001 <= 1)

),

);

doDiscriminant =

Expr(

da = dt << Discriminant(

X( :Site Type ),

Y(

:

:Name( "366" ),

:Name( "367" ),

:Name( "368" ),

:Name( "369" ),

:Name( "370" ),

:Name( "371" ),

:Name( "372" ),

:Name( "373" ),

:Name( "374" ),

:Name( "375" ),

:Name( "376" ),

:Name( "377" ),

:Name( "378" ),

:Name( "379" ),

:Name( "380" ),

:Name( "381" ),

:Name( "382" ),

)

);

for(i=1, i<=10, i++,

doDiscriminant;

da << StepwiseVariableSelection;

for(k=1, k<=i, k++, da << stepForward);

da << applyThisModel;

daRep = da << Report;

daRep["Score Summaries"]<<select;

daRep["Score Summaries"] << Journal("test1");

Current Journal() << Append(text Box("Test1"));

daRep << closeWindow;

);

Current Journal() << Save Journal ("Rand001") [Ideally, this name updates with each iteration]

The discriminant analysis is on loop for a certain amount of stepwise variable selection iterations (10): What I want is for each 1-10 iteration of the discriminant analysis to occur for each iteration of a data filter. The data filter is set to read a column (Rand001, Rand002,...Rand100), and than exclude specific rows based on the content of the column.

Also, I wasn't able to find a simple way to save journals/text files iteratively with a name that updates per-iteration, so that is another piece of useful advice here.

Thank you!

5 REPLIES

Super User

Joined:

Mar 17, 2015

I don't think I answered all of your questions but this might put you on the right track.

dt = open("$SAMPLE_DATA\big class.jmp"); dt<<New Column("Rand001", Formula(Random Integer(0,1))); dt<<New Column("Rand002", Formula(Random Integer(0,1))); dt<<New Column("Rand003", Formula(Random Integer(0,1))); for(i = 1, i<=3, i++, col = Column("Rand00"||char(i)); da = dt << Discriminant( X( :sex ), Y( :height ), Where(col[row()] == 1), Show Classification Counts( 1 ), Biplot Ray Position( [0, 15.5914608488282, 1.5] ) ); da << Title("Rand00"||char(i)); da << Redraw; da << Journal; da << Close Window(); ) I would use either the where statement or the local data filter. hope it's at least somewhat useful. Community Trekker Joined: Feb 10, 2015 Thank you for that help, it has given me some insight on my next steps. Though, what I was looking for was to iteratively apply a data filter, not iteratively run the DA. Example, here is a sample code: dt = Current Data Table(); col= Column("Rand00"||char(i)); dt = open("$SAMPLE_DATA\big class.jmp");

dt<<New Column("Rand001", Formula(Random Integer(0,1)));

dt<<New Column("Rand002", Formula(Random Integer(0,1)));

dt<<New Column("Rand003", Formula(Random Integer(0,1)));

doDataFilter =

col= Column("Rand00"||char(i));

dt << Data Filter(

Mode(Include(1)),

Inverse,

Columns("Rand00"||char(i)),

Where(:col == {1})

//Where(:"Rand00"||char(i) > 0.875 & :"Rand00"||char(i) =1)

),

);

for(i=1, i<=3, i++, doDataFilter;

col= Column("Rand00"||char(i)););

The WHERE clause has not been correctly specified. in access or evaluation of 'Equal' , :col ==  /*###*/{1} /*###*/

How can I write the WHERE clause to include the 'i' character, so that the filter moves along the designated rows with each iteration?

Super User

Joined:

Mar 17, 2015

I guess then what's the purpose of excluding?  If you're just excluding so you get a different graph every time, you can do that with the Where.

other than that, you shouldn't column scope col.

:col == {1} // won't work

col == 1 // will work

Community Trekker

Joined:

Feb 10, 2015

Vince,

Thank you for this information, I understand that my current method likely isn't the most efficient. If the Where segment of your suggested code will indeed exclude rows with cell-values of 1 for the respected Rand column, what is the syntax? This will solve my problem!

Super User

Joined:

Jun 23, 2011

Not sure what you aim at in the end, but this is one way to iteratively update the data filter:

dt = Open("\$SAMPLE_DATA\big class.jmp");

dt << New Column("Rand001", Formula(Random Uniform(0, 1)));

dt << New Column("Rand002", Formula(Random Uniform(0, 1)));

dt << New Column("Rand003", Formula(Random Uniform(0, 1)));

// "Blank" data filter object

df = dt << Data Filter(Mode(Include(1)), Inverse);

For(i = 1, i <= 3, i++,

col = Column("Rand00" || Char(i)); // Dynamic col variable

df << delete all; //Remove old columns from df