Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

- JMP User Community
- :
- Discussions
- :
- Adding augmented data points to a data table

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Adding augmented data points to a data table

Created:
Jul 24, 2019 5:49 AM
| Last Modified: Jul 24, 2019 6:37 AM
(859 views)

Hello JMP community,

I currently am trying to write a script that opens an existing data table which Contains an "X1" column and a "Y1" column with 10 rows. The script then predicts Y1 in a new column, makes a residual column (Predicted Y1 - Y1), and then makes a predicted residual column. From there, I want the script to add a certain amount of "made up" X1/Y1 values to the existing data table, in hopes of getting the process to automate. The following code is what I believe is responsible for adding the data points to the data table. The problem is that the following script doesn't add an accurate list of X1 values (the X1 values is simply every integer between 1-10, and when I run the following code I usually only get integers 1 and 10).

```
d = DOE( Augment Design, X( :X1 ), Y( :Y1 )/*, {Make Design, Make Table}*/ );
d << Make Design;
dt3 = d << Make Table;
dt2 << concatenate( dt3, Append to first table( 1 ) );
```

Here is my entire script if it helps any:

```
// 1. Creates X1 column, and corresponding Y1
Close All( Data Tables, No Save );
sourceDT = Open( "SH2.jmp" );
dt2 = sourceDT << Subset( Columns(), Rows(), Output Table Name( "Fixed.jmp" ) );
// 2. Creates predicted Ystat column, saves formulas into same data table (first iteration)
PredictedY1 = dt2 << Neural(
Y( :Y1 ),
X( :X1 ),
Informative Missing( 0 ),
Validation Method( "Holdback", 0.3333 ),
Fit( NTanH( 3 ) )
);
PredictedY1 << Save Profile Formulas;
// 3. Adds residual column
dt2 << New Column( "Residual 1", Formula( Abs( :PredictedY1 - :Y1 ) ) );
// 4. Creates and predicts residual column
PredictedResidual1 = dt2 << Neural(
Y( :Residual 1 ),
X( :X1 ),
Informative Missing( 0 ),
Validation Method( "Holdback", 0.3333 ),
Fit( NTanH( 3 ) )
);
PredictedResidual1 << Save Profile Formulas;
// Iteration 2
d = DOE( Augment Design, X( :X1 ), Y( :Y1 )/*, {Make Design, Make Table}*/ );
d << Make Design;
dt3 = d << Make Table;
dt2 << concatenate( dt3, Append to first table( 1 ) );
PredictedY2 = dt2 << Neural(
Y( :Y1 ),
X( :X1 ),
Informative Missing( 0 ),
Validation Method( "Holdback", 0.3333 ),
Fit( NTanH( 3 ) )
);
PredictedY2 << Save Profile Formulas;
// 3. Adds residual column
dt2 << New Column( "Residual 2", Formula( Abs( :PredictedY1 2 - :Y1 ) ) );
// 4. Creates and predicts residual column
PredictedResidual2 = dt2 << Neural(
Y( :Residual 2 ),
X( :X1 ),
Informative Missing( 0 ),
Validation Method( "Holdback", 0.3333 ),
Fit( NTanH( 3 ) )
);
PredictedResidual2 << Save Profile Formulas;
Column( dt2, "X1" ) << Suppress Formula Eval();
Column( dt2, "Y1" ) << Suppress Formula Eval();
dt2 << Select where( :Residual 1 <= 3 );
dt2 << delete rows;
```

Thanks,

Kyle

3 REPLIES 3

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Adding augmented data points to a data table

I am not quite sure what you want. My interpretation is that you want to append new rows to the previous rows. Augment Design creates new treatments and the Make Table message will output the new combined table. It creates the new rows that are optimal for the specified model and the given rows.

I think that you want to create a second data table with the levels you determine and concatenate them to the existing rows. Alternatively, you can iteratively send the << Add Rows() message to the first data table to append the new levels.

Learn it once, use it forever!

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Adding augmented data points to a data table

I apologize for any confusion. You are correct that I want to create a second data table with the levels I determine and concatenate them to the existing rows. The problem is that the second data table creates a table with values explicitly either 1 or 10. I want the second data table to have values randomly distributed between 1 and 10 (which is the range of values that the original data table has). Augment Design may not be the best way to do this, but I am very new to JSL so i'm not sure of how to proceed

Thanks

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Adding augmented data points to a data table

You can still use either method I suggested: create a second data table and concatenate it or directly add rows to original data table.

Sketch of idea:

`dt2 = New Table( "Augmentation Runs",`

Add Rows( 10 ),

New Column( "X1", Numeric, Continuous, Values( J( 10, 1, Random Uniform() ) )

);

dt << Concatenate( dt2 );

Alternatively:

```
For( I = 1, I < 11, I++,
dt << Add Rows( { X1 = Random Uniform() } );
);
```

Learn it once, use it forever!

Article Labels

There are no labels assigned to this post.