Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

- JMP User Community
- :
- Discussions
- :
- Model verification/validation

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Model verification/validation

Nov 27, 2019 1:28 AM
(1466 views)

Dear all,

I generated a model from 20 data generated with a DoE.

I also run in parallel 20 runs designed with a LHS on the same design space.

I would like to compare the observed responses obtained with the LHS to the predicted responses obtained with the DoE.

Which kind of procedure would you suggest to validate/verify the initial DoE model ?

I used equivalence testing but it looks as a stringent test (I mean, a model seems to be highly performing to pass such a test).

Please note that I do not have JMP Pro,

Many thanks,

Hugo

5 REPLIES 5

Highlighted
##

It's not clear to me what an 'LHS' is...but I'd start simple with respect to comparing the results. I'll presume an 'LHS' is some sort of empirical investigation of the same treatment combination for treatment combination corresponding to the treatment combinations used to make the predictions across the original DOE's design space? If that's the case, I'd start simple. Maybe just a histogram of the difference between the predicted values and the 'LHS' values. Then maybe a scatter plot (Graph Builder or Fit Y by X) of the predicted vs. LHS values, with the predicted values on one axis, the LHS on the other.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Model verification/validation

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Model verification/validation

LHS states for Latin Hypercube Sampling and is a space filling design. I used the same factors as for the DoE to design this LHS design. Thanks for the suggestions.

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Model verification/validation

You might use a matched pairs t-test for a significant difference between predicted response and observed response. That is, use the original model to predict the response for each run in the Latin hyper-square design. Your data table will use two data columns for this analysis. Enter them as predicted and observed in the Y role for Matched Pairs launch.

Learn it once, use it forever!

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Model verification/validation

Just curious: what is the difference, if any, between the models obtained from the two DOEs for the same factors?

Learn it once, use it forever!

Highlighted
##

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Re: Model verification/validation

Many thanks for the tip. Actually, there are differences (but all rather small differences).

Article Labels

There are no labels assigned to this post.