Value selection and copy into new columns

[#1]

Hey,

I´m trying to automatize a (seemingly fairly easy) value selection, column creation and copy/paste process, however, I cannot find the right script compilation for it.

1) I need to create 3 new columns adding a different selection of values from one same source column to each of those 3 new columns and create a subset of each selection when all columns are filled in.

2) The name of the subsets should correspond to the header of the original source file plus a new appendix.

Be aware:

The location of columns and the amount of rows varies in the data files I have, however, the names are all the same.

I tried these two scripts which of course are not correct but I do not know what to change exactly to make it work. Any suggestions?

Version 1:

Names Default To Here( 1 );
dt = Current Data Table();
//Select values to copy into new column
vals 1 = Select Where (:p-value (two-sided test)2 <= 0,09); //it seems that the "2" in the column name is confusing, but unfortunately, it would take a long time to go through all tables and change the column name
 // Put these in a new column 1
dt = NewColumn("sign_0,09", Numeric, Continuous, Values(vals1));
Wait( 5 );
//Column 2
vals 2 = Select Where (:p-value (two-sided test)2 <= 0,05);
dt<<get selected rows
dt = NewColumn("sign_0,05", Numeric, Continuous, Values(vals2));
Wait( 5 );
//Column 3
vals 3 = Select Where (:p-value (two-sided test)2 <= 0,01);
dt<<get selected rows
dt = NewColumn("sign_0,01", Numeric, Continuous, Values(vals3));
//Create subsets
dt << Select where :sign_0,09 != "";
dt<< Get selected rows
//the new name should be automatically given, too, is that possible?
dt << Subset( Output Table( "name as in original file plus _0,09 at the end" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select where :sign_0,05 != "";
dt<< Get selected rows
dt << Subset( Output Table( "name as in original file plus _0,05 at the end" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select where :sign_0,01 != "";
dt<< Get selected rows
dt << Subset( Output Table( "name as in original file plus _0,01 at the end" ), Selected Rows( 1 ), selected columns( 0 ) );

Version 2:

Names Default To Here( 1 );
dt = Current Data Table();

dt << :p-value (two-sided test)2 = [1,?]; //the end row varies and must be found automatically
results << New Column("sign_0,09", Numerical, Continuous);
For( i = 1, i <= NItems( "p-value (two-sided test)2"), i++,
    results << Select Where (:p-value (two-sided test)2 <= 0,09) );
    selRows = results << getSelectedRows;
 AsColumn( results, "sign_0,09" )[selRows] = "p-value (two-sided test)2"[i];
//same process for 2 other new columns
dt<<Subset
dt << Select where :sign_0,09 != "";
dt<< Get selected rows
//the new name should be automatically given, too, is that possible?
dt << Subset( Output Table( "name as in original file plus _0,09 at the end" ), Selected Rows( 1 ), selected columns( 0 ) );
//same process for other 2 columns

I would be very happy to find a solution as in the threads so far there is often a copy/paste into new data tables, however, "never" into a new column, and mostly it´s related to a string selection it seems. Not sure if the selection works the same for numerical data.

Thanks for any help!

[#2]

I see this approach but there is a simpler way that does involve changing the selected row state. Instead of this:

SR1 = dt << Select Where(:Name("p-value (two-sided test) 2") <= 0.09) << Get Selected Rows; 
// You can edit this to conform to european convention of 0,09
dt << Clear Select; 

You can just use this:

SR1 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.09) 
// You can edit this to conform to european convention of 0,09

This way you don't disturb the current row seletion.

[#3]

@uday_guntupalli

I ran the script with my data (your version and with the exchange of the suggestion of @Mark_Bailey) and it works to create the columns and to get the selected data into the new column.

However, the last part (see below) to show the selected rows does not work. I adjusted it with a Select Where and Create Subset.

/* Step 3 */
// We already have the rows that meet the conditions 
Show(SR1); 
Show(SR2); 
Show(SR3); 
/* End of Step 3 */ 

/* Step 3 */
dt << Select Where (Name("p-value (two-sided test)2") <= 0.09);
dt << Subset( Output Table( "new_0,09" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.05);
dt << Subset( Output Table( "new_0,05" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.01);
dt << Subset( Output Table( "new_0,01" ), Selected Rows( 1 ), selected columns( 0 ) );
/* End of Step 3 */  

[#4]

Hey, here is the full version without the Show functions as those were not of any use in my case.

Thanky for your help!

dt = Current Data Table(); // Assuming the data table is open

/* Step 1 - add 3 columns  */
dt << New Column("sign_0,09",Numeric,Continuous)
   << New Column("sign_0,05",Numeric,Continuous)
   << New Column("sign_0,01",Numeric,Continuous);

// Getting the values of column of interest
ColVals = dt:Name("p-value (two-sided test) 2") << Get Values;

// Set up a dummy matrix
Dummy = J(NRows(dt),1,.);

// All Rows
AllRows = Index(1,NRows(dt),1);

/* Step 2 - Select from the column based on conditions */
/* This could also be replaced by loop */
SR1 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.09);
//do not change "." to "," as the programme uses "." for background calculation
Dummy[SR1,0] = ColVals[SR1];

dt:Name("sign_0,09") << Set Values(Dummy);

// Re-initialize dummy matrix
Dummy = J(NRows(dt),1,.);

SR2 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.05);
Dummy[SR2,0] = ColVals[SR2];

dt:Name("sign_0,05") << Set Values(Dummy);

// Re-initialize dummy matrix
Dummy = J(NRows(dt),1,.);

SR3 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.01);
Dummy[SR3,0] = ColVals[SR3];

dt:Name("sign_0,01") << Set Values(Dummy);
/* End of Step 2 */

/* Step 3 */
dt << Select Where (Name("p-value (two-sided test)2") <= 0.09);
dt << Subset( Output Table( "new_0,09" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.05);
dt << Subset( Output Table( "new_0,05" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.01);
dt << Subset( Output Table( "new_0,01" ), Selected Rows( 1 ), selected columns( 0 ) );
/* End of Step 3 */

[#5 · Accepted solution]

Hey, here is the full version without the Show functions as those were not of any use in my case.

Thanks for your help!

dt = Current Data Table(); // Assuming the data table is open

/* Step 1 - add 3 columns  */
dt << New Column("sign_0,09",Numeric,Continuous)
   << New Column("sign_0,05",Numeric,Continuous)
   << New Column("sign_0,01",Numeric,Continuous);

// Getting the values of column of interest
ColVals = dt:Name("p-value (two-sided test) 2") << Get Values;

// Set up a dummy matrix
Dummy = J(NRows(dt),1,.);

// All Rows
AllRows = Index(1,NRows(dt),1);

/* Step 2 - Select from the column based on conditions */
/* This could also be replaced by loop */
SR1 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.09);
//do not change "." to "," as the programme uses "." for background calculation
Dummy[SR1,0] = ColVals[SR1];

dt:Name("sign_0,09") << Set Values(Dummy);

// Re-initialize dummy matrix
Dummy = J(NRows(dt),1,.);

SR2 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.05);
Dummy[SR2,0] = ColVals[SR2];

dt:Name("sign_0,05") << Set Values(Dummy);

// Re-initialize dummy matrix
Dummy = J(NRows(dt),1,.);

SR3 = dt << Get Rows Where(:Name("p-value (two-sided test) 2") <= 0.01);
Dummy[SR3,0] = ColVals[SR3];

dt:Name("sign_0,01") << Set Values(Dummy);
/* End of Step 2 */

/* Step 3 */
dt << Select Where (Name("p-value (two-sided test)2") <= 0.09);
dt << Subset( Output Table( "new_0,09" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.05);
dt << Subset( Output Table( "new_0,05" ), Selected Rows( 1 ), selected columns( 0 ) );
dt << Select Where (Name("p-value (two-sided test)2") <= 0.01);
dt << Subset( Output Table( "new_0,01" ), Selected Rows( 1 ), selected columns( 0 ) );
/* End of Step 3 */