Subscribe Bookmark RSS Feed

Iteration Questions: Data Filtering, and File Saving-Naming

ajgarnello

Community Trekker

Joined:

Feb 10, 2015

Hello,

I am wondering how to setup a code that iteratively runs an analysis according to a specific data filter, and then repeat that analysis for a new data filter. The data filter is set to exclude rows based on the presence of a 1 or 0 in a column, and the columns are in sequential order. Here is an example script:

dt = Open("Hyperspectral_DataCleanedOrdered.jmp");

dt << Data Filter(

  Mode(Include(1)),

  Inverse,

  Add Filter(

  Columns(Rand001),

  Where(:Rand001 >= 0.875 & :Rand001 <= 1)

  ),

);

doDiscriminant =

Expr(

  da = dt << Discriminant(

  X( :Site Type ),

  Y(

  :

  :Name( "366" ),

  :Name( "367" ),

  :Name( "368" ),

  :Name( "369" ),

  :Name( "370" ),

  :Name( "371" ),

  :Name( "372" ),

  :Name( "373" ),

  :Name( "374" ),

  :Name( "375" ),

  :Name( "376" ),

  :Name( "377" ),

  :Name( "378" ),

  :Name( "379" ),

  :Name( "380" ),

  :Name( "381" ),

  :Name( "382" ),

  )

);

for(i=1, i<=10, i++,

  doDiscriminant;

  da << StepwiseVariableSelection;

  for(k=1, k<=i, k++, da << stepForward);

  da << applyThisModel;

  daRep = da << Report;

  daRep["Score Summaries"]<<select;

  daRep["Score Summaries"] << Journal("test1");

  Current Journal() << Append(text Box("Test1"));

  daRep << closeWindow;

);

Current Journal() << Save Journal ("Rand001") [Ideally, this name updates with each iteration]

The discriminant analysis is on loop for a certain amount of stepwise variable selection iterations (10): What I want is for each 1-10 iteration of the discriminant analysis to occur for each iteration of a data filter. The data filter is set to read a column (Rand001, Rand002,...Rand100), and than exclude specific rows based on the content of the column.

Also, I wasn't able to find a simple way to save journals/text files iteratively with a name that updates per-iteration, so that is another piece of useful advice here.

Thank you!

5 REPLIES
vince_faller

Super User

Joined:

Mar 17, 2015

I don't think I answered all of your questions but this might put you on the right track.

dt = open("$SAMPLE_DATA\big class.jmp");

dt<<New Column("Rand001", Formula(Random Integer(0,1)));

dt<<New Column("Rand002", Formula(Random Integer(0,1)));

dt<<New Column("Rand003", Formula(Random Integer(0,1)));

for(i = 1, i<=3, i++,

      col = Column("Rand00"||char(i));

      da = dt << Discriminant(

            X( :sex ),

            Y( :height ),

            Where(col[row()] == 1),

            Show Classification Counts( 1 ),

            Biplot Ray Position( [0, 15.5914608488282, 1.5] )

      );

      da << Title("Rand00"||char(i));

      da << Redraw;

      da << Journal;

      da << Close Window();

)

I would use either the where statement or the local data filter.

hope it's at least somewhat useful.

ajgarnello

Community Trekker

Joined:

Feb 10, 2015

Thank you for that help, it has given me some insight on my next steps. Though, what I was looking for was to iteratively apply a data filter, not iteratively run the DA. Example, here is a sample code:

dt = Current Data Table();

col= Column("Rand00"||char(i));


dt = open("$SAMPLE_DATA\big class.jmp");

dt<<New Column("Rand001", Formula(Random Integer(0,1)));

dt<<New Column("Rand002", Formula(Random Integer(0,1)));

dt<<New Column("Rand003", Formula(Random Integer(0,1)));

doDataFilter =

  col= Column("Rand00"||char(i));

  dt << Data Filter(

  Mode(Include(1)),

  Inverse,

  Add Filter(

  Columns("Rand00"||char(i)),

  Where(:col == {1})

//Where(:"Rand00"||char(i) > 0.875 & :"Rand00"||char(i) =1)

),

);

for(i=1, i<=3, i++, doDataFilter;

col= Column("Rand00"||char(i)););


This code reads the error:

The WHERE clause has not been correctly specified. in access or evaluation of 'Equal' , :col ==  /*###*/{1} /*###*/


How can I write the WHERE clause to include the 'i' character, so that the filter moves along the designated rows with each iteration?


vince_faller

Super User

Joined:

Mar 17, 2015

I guess then what's the purpose of excluding?  If you're just excluding so you get a different graph every time, you can do that with the Where. 

other than that, you shouldn't column scope col.

:col == {1} // won't work

col == 1 // will work

ajgarnello

Community Trekker

Joined:

Feb 10, 2015

Vince,

Thank you for this information, I understand that my current method likely isn't the most efficient. If the Where segment of your suggested code will indeed exclude rows with cell-values of 1 for the respected Rand column, what is the syntax? This will solve my problem!

ms

Super User

Joined:

Jun 23, 2011

Not sure what you aim at in the end, but this is one way to iteratively update the data filter:

dt = Open("$SAMPLE_DATA\big class.jmp");

dt << New Column("Rand001", Formula(Random Uniform(0, 1)));

dt << New Column("Rand002", Formula(Random Uniform(0, 1)));

dt << New Column("Rand003", Formula(Random Uniform(0, 1)));

// "Blank" data filter object

df = dt << Data Filter(Mode(Include(1)), Inverse);

For(i = 1, i <= 3, i++,

    col = Column("Rand00" || Char(i)); // Dynamic col variable

    df << delete all; //Remove old columns from df

    df << add filter(Columns(col), Where(col > 0.875));// Add current column

    Wait(2); //Delay just for demo purposes

);

But the approach of Vince above, to include a where clause in the analysis platform instead of using a data filter may make more sense here.