is there a script I can use to automate validation testing of my multiple regression model in JMP? (not JMP Pro)

[#1]

Hi - Is it possible to use a script or other approach to automate validation testing of my multiple regression model?  Currently, I am manually removing a subset of rows from my original model, using the balance of rows for regression, and then using the subset to evaluate how well the model performed.  I then repeat this numerous times.  It is very time consuming, especially given the fact that I now need to change my original model slightly.  I realize that JMP Pro will do this but currently do not have access to JMP Pro.  Thanks in advance!

[#2]

Yes you could write a script that subsets you data table, perfroms your regression analysis and saves and applies the prediction column to your held back set. To do this just exectute the steps manually, capture all the script lines from the log. Glue them together in a script file and with a little tweaking you are 90% there. Good luck!

[#3]

Thank you Pauldeen.   So my work flow would be:

1) open orig table in JMP (original table with 164 rows)

2) create random subset of rows from orig table

     2.a.   randomly pull 16 rows from orig table and "link" those rows to the orig table

     2.b.   save newly created subset table with a unique name (e.g. "TEST 1 - SUBSET")

     2.c.   highlight/select all 16 rows in subset table 

     2.d.   click on orig table and delete the highlighted 16 rows

     2.e.   save orig table (which now contains 164-16 rows) to a name similar to 2.b. (e.g. "TEST 1 - MAIN")

3) open TEST 1 - MAIN

4) analyze, fit model, set parameters for y and x (I use "recall" after the first run), run, save

5) in the resulting "fit least squares" window click red "response" triangle, save columns, prediction formula

6) open TEST 1 - SUBSET

7) add three new columns:  1) "test1_prediction".....2) "is_test1_prediction > 410?".....3) "difference test1_pred > 410 minus actual > 410"

copy TEST 1 - MAIN formula in "prediction formula" column to TEST 1 - SUBSET  column titled "test1_prediction"   

9) perform conditional test on the results from and place in column titled "is_test1_prediction > 410?"

10) add formula to "difference test1_pred > 410 minus actual > 410" to subtract results of 9) from an existing column in TEST 1 - MAIN

11) save TEST 1 - SUBSET

12) repeat steps 1) through 11)

I think I understand how to do the "analyze" steps and save each to a script, but what I do not understand is how to do the other steps via script, such as load a file, create new columns (with specified names), copy formula from one column in one file to a column in another file, and save tables (with names corresponding to the sequentially increasing test number).    Any suggestions?

[#4]

It sounds like you need to start learning how to script. There are enough example scripts in this community that show you how to write file open and column copy scripts.There is also the scripting guide under the help menu and there are example scripts in the sample folders under help. The most important tip with JMP scripting is that you should do as little of it as possible, under the red triangles you can copy scripts, in the log file you will see script snippets, use those. Aggregate them in a big script file and it should not be to hard.

[#5]

Thank you Pauldeen.  I'll have a look.  Since I don't often use scripting, hopefully the learning curve will be time well spent!  (I am with a State Agency and do not have access to hire your skills - thank you though for the heads up on your availability.)

[#6]

Hi @learning_JSL ,

  Learning JSL will help you with your work and help to speed things up. Scripting is pretty straightforward and rather intuitive. One thing you'll definitely want to do is use the Help > Scripting Index to help you. In fact, there is a command called Make Validation Columns() which will make your training, validation, and test data sets in your data table. Under any of JMP's modeling platforms, all you need to do is cast the validation column in its role and JMP knows how to utilize that data in the modeling process. Once you get the right fit that you're after you can save the prediction formula to the data table.

Hope this helps!,

DS

[#7]

Thank you for your reply DS.  I'll give it a try and will watch for the "Make Validation Columns" command as well.