Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- JMP Discovery Summit Series
- :
- Discovery Summit Europe 2018 Presentations
- :
- Model Validation Strategies for Designed Experiments Using Bootstrapping Techniq...

Feb 14, 2018 4:54 AM

Level: Intermediate**Philip Ramsey, Principal Lecturer, University of New Hampshire; and Owner, North Haven Group**

**Chris Gotwalt, JMP Director of Statistical Research and Development, SAS**

There are two different goals to statistical modeling: explanation and prediction. Explanatory models often predict poorly (Shmueli, 2010). Often analyses of designed experiments (DOE) are explanatory, yet the experimental goals are prediction. DOE is a best practice for product and process development where one predicts future performance. Predictive modeling requires partitioning the data into training and validation sets where the validation set is used to assess predictive models. Most DOEs have insufficient observations to form a validation set precluding direct assessment of prediction performance. We demonstrate a “balanced auto-validation” technique using the original data to create two copies of that data, one a training set and the other a validation set. The sets differ in row weightings. The weights are Dirichlet distributed and “balanced;” observations contributing more to the training data contribute less to the validation set (and vice versa). The approach only requires copying the data and creating a formula column for weights. The technique is general, allowing one to apply predictive modeling techniques to smaller data sets common to laboratory and manufacturing studies. Two biopharma process development case studies are used for demonstration. Both cases have large validation sets combined with definitive screening designs. JMP is used to demonstrate analyses.

Comments

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

03-23-2018
12:38 PM

This was a really interesting presentation. Thanks for sharing the files. I am trying to reproduce this on one of my DOEs.

However I got stuck at step 10 from the slides: Calculate the proportion non zero. That option is not available for me (see picture below)?? Does anyone know why that might be the case? I am using JMP Pro 14EA.

I have got around this by using N Zero instead and making an extra calculation to get the proportion non zero. Works nicely thanks!!

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

03-24-2018
09:59 AM

Yes, the easy solution is to use the NZero function (number of times 0)

and then you can calculate the proportion zero or nonzero. If you look

at the case studies, then you will see I used the NZero function to

create various weighting schemes.

Phil Ramsey

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

03-26-2018
05:45 AM

Thanks Phil. I have now downloaded JMP14, and the option for proportion on zero is now included :)

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

10-30-2018
02:06 PM

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

10-31-2018
06:30 PM

Really nice to hear you are having fun with the autovalidation method. The method is very general and can easily be applied to observational or DOE data and I encourage you to try. The method is done exactly the same way whether observational or DOE. I also have applied it to small observational datasets and have gotten very useful results in terms of finding which effects appear most important.

Phil Ramsey

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

10-31-2018
06:37 PM

I am having so much fun with it – it is distracting me from trying out the Functional Data Explorer – which I want to try next. It takes a lot of time to run the model simulations – but I think it is well worth it for what I am doing and helping us identify important factors for further experimentation!

Thank you!

Patti

- Mark as Read
- Mark as New
- Bookmark
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

11-01-2018
09:53 AM

Really pleased that you are finding autovalidation useful and fun .

Phil

- Article History
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Email to a Friend
- Printer Friendly Page
- Report Inappropriate Content