Choose Language Hide Translation Bar

OMARS Designs Add-in: A Gateway to a New Family of Orthogonal RSDs (2022-EU-30MP-1051)

Hadley Myers, JMP Systems Engineer, SAS
Phil Kay, Learning Manager, Global Enablement, JMP

 

The definitive screening design (DSD) is almost certainly the 21st century's most exciting and useful innovation in design of experiments (DOE). As a screening design, the DSD offers unique properties for a much smaller number of a runs. And, if only a half or fewer of the factors are active, the DSD gives you the ability to fit the full response surface model. However, if your objective is to estimate the full response surface model for most or all of your factors, a DSD is inappropriate. In those instances, larger optimal designs or central composite designs (CCDs) are the preferred choices. Orthogonal minimally aliased response surface (OMARS) designs are a new family of response surface design (RSD) that bridges the gap between the small, efficient DSD and the large, high-powered CCD.

In this presentation, we introduce OMARS designs by way of a case study comparison with other designs. We also demonstrate how JMP users can create and evaluate OMARS designs against DSDs and classical RSDs in an easy-to-use add-in that will help you to select the right design for your specific application.

 

 

 

Okay, so welcome.

I'm Phil Kay, and I'm joined by Hadley Myers.

We're going to talk about O MARS Designs,

this new family of design of experiments

and an ad d-in that gives you a gateway into that world.

I'll start with an introduction,

and I'm going to give you a motivating case study.

I'm going to talk about how these OMARS Designs

bridged the gap between our small, efficient,

Definitive Screening Designs that we're all familiar with,

and the larger, high- powered, more traditional r esponse surface designs

that you might know of.

Then I'll pass over to Hadley

and he'll talk about how you, as a JMP user, can create and evaluate

different OMARS Designs with an add-in that he's been working on.

These O MAR Designs,

they come from a paper by Jose Nunez Ares and Peter Goos.

I'll just show you that briefly.

They've worked through the enumeration of thousands of such designs,

and we'll introduce you to what these designs look like.

I've got a motivating case study to begin.

This is from a published case study.

This was published in the Journal of Clinical Chemistry.

It's a response surface design.

It's about optimizing clinical chemical methods,

so an assay, in this case.

The objective was to optimize an assay method,

maximizing the response,

which is called Elevated Serum 30 degrees C.

They had six factors, each of which was a quantity

of a different reagent,

and they took a traditional approach.

This was done quite some time ago.

They generated a Central Composite Design,

which is a very traditional response surface design for optimization

with 48-runs, which includes four center points.

I've used this to motivate the use of O MARS Designs.

As an alternative to this 48- run design,

I generated a 17- run Definitive Screening Designs for those six factors.

I also generated an alternative 31- run O MARS Design.

I took the model from the original 48-run experiment.

So use that data, fit a model, and use that to simulate the responses

that we might expect for the Definitive Screening Design and fully OMARS Design.

We added an appropriate amount of noise

to that to give us a realistic response simulation.

What you're going to see through this example

is that the Definitive Screening Design is effective at what it should do,

which is finding the most important factors.

The O MARS Design enables us to optimize the process by identifying and estimating

all of the important effects from the response surface model.

In this way, we are saying that these O MARS Designs,

you can think of them as bridging the gap between Definitive Screening Designs

and the traditional response surface method designs,

like the Central Composite Design.

Here is that Central Composite Design.

Here are our six factors, and this is our response of interest.

These are some of the models that we fit, and we're comparing.

It's a traditional face- centered, Central Composite Design.

This is just three of the factors visualized.

You can see we've got our axial points here

on the face of the cube that's described by the factor ranges.

Those kind of designs are very good.

They've got lots of nice properties

in terms of the correlations between effects.

You can see lots of white space here, which means zero correlations,

orthogonal effects.

They're not so great with the quadratic effects.

There's fairly strong correlations between all of our quadratic effects

on one another, which does reduce the power of our ability to estimate

these quadratic effects.

Nevertheless, we can fit a good model to that.

This is the model fit to that original data.

We identified that there really are four critical factors out of the six

and there are various higher order terms as well that are important.

Really the pH, this P 5P OG, and MDH are very important.

The L-a spartic acid and this Tris buffer are much less important.

We can build a good model using that design.

It's quite a big expensive design, though, 48-runs.

What would our alternatives be?

Well, Definitive Screen Designs are obviously very good

for screening these kind of situations, screening for the important factors.

I've generated using the same factors, same factor ranges,

a 17-run Definitive Screening Design for those six factors.

A gain, I've simulated the response data there based on the model

from the published data from the big experiment there.

The definitive screen design does what it's supposed to do.

It finds that we've got these four important factors,

the pH, P5 P, OG, MDH.

It's identified those and it's been able

to identify some of the higher order effects that are important.

Now at this stage, what you could do is augment.

Screening design is all about screening for the important factors

and then in the next step of the sequence, experimental sequence,

we can augment to learn more about the higher order effects,

the higher order terms for the response surface model.

What I'm going to show you here, though, is an alternative approach

we could have taken.

Here is an experimental design with 31 runs.

Again, same six factors, the same factor ranges.

You'll notice that it is a three- level design.

For each factor, we've got settings at three levels.

It's a response surface design.

If we compare it using the compare designs platform,

then we can compare those two designs.

The Definitive Screening Design here in blue,

we're looking at the powers versus the 31 run OMARS Design.

Well, it's not a surprise that the 31-run design has higher power.

Generally, we've got more runs, so we would expect that.

We can see significantly higher power for these quadratic effects, though.

Another thing to look at that might be of interest

is the color map on correlations.

Here's our 17-run Definitive Screening Design,

and you might recognize that color map,

if you know anything about Definitive Screening Designs then.

This color map is really a key.

It demonstrates a key property of Definitive Screening Designs,

which is that all of our main effects are orthogonal to one another,

and the main effects are also orthogonal to the second order effects,

the quadratics and the two-factor interactions.

That's what all that white space there means.

Then within the higher order terms,

there is some degree of correlation, but no complete correlation, no aliasing.

We are always able to estimate some of these higher order terms,

and those higher order terms are, at least, orthogonal,

completely separately estimated from the main effects factors,

the fact ors main effects rather.

Now if we look at this 31-run O MARS Design,

you can see its got similar properties.

Again, we've got orthogonal main effects,

and those main effects are orthogonal to the second order effects.

You can see we've got lower correlation between the quadratic effects,

for example.

Overall with the two- factor interactions as well,

there are lower correlations.

Why are these things called OMARS?

Well, OMARS stands for Orthogonal Minimally Aliased R esponse Surface

designs.

Again, we've got orthogonal main effects,

and we've got minimal aliasing between our second order effects as well,

and it's a response surface design.

In fact, both of these,

both the Definitive Screening Design and the 31- run design are OMARS.

They are both Orthogonal Minimally Aliased Response Surface Designs,

so DSDs are a subset of OMARS.

How well does this perform?

What I've done is I fitted a model to that simulated data.

Again, I simulated the response data for our 31- run O MARS Design,

and I've compared that model against the 17-run Definitive Screening Design.

17-run Definitive Screening Design is doing a reasonable job

of predicting the actual data.

Here, we're comparing how well our two models

from the Definitive Screening Design and the OMARS Design,

how well they fit against the actual data from the 48-run published example.

You can see a much improved model

with the 31- run O MARS Design, as we might expect.

In fact, the 31- run O MARS Design

has identified correctly the higher order terms that are important,

as well as identifying the important factor effects,

which was pretty much

all the Definitive Screening Design was able to do.

Again, just to reiterate, what we're showing here

is that these OMARS Designs

are really an extension of Definitive Screening Designs,

and they are a bridge between that small, efficient Definitive Screening Design

and the larger traditional response surface designs.

At this point, I'll hand you over to Hadley,

who's going to show you more about an add-in that he's created

that will enable you to actually explore this new class of designs for yourself.

All right. Thank you very much, Phil.

Hello to everyone watching this online, wherever you are.

Thank you very much for clicking on this talk.

Before I take you through the add-in to show you how you can use it

to generate these designs and select the best one for you,

I'd like to say that the add-in itself includes 7,886 files,

each one containing a design,

where the main effects are orthogonal to each other into the higher order terms.

The add-in not only gives you access to these 7,886 new designs,

but it also gives you access to all of these designs

with an added center point.

How can we select from among these

almost 16,000 designs the best one for us in our situation,

while the add-in provides us an interface to allow us to do that?

I'll show you how that works.

Right now, the add-in is called OMARS Explorer.

What it will allow us to do

is first indicate the number of factors that we have,

and the add-in at this moment has the ability to generate designs

for five, six, or seven continuous factors.

We can write the maximum number of runs that we can afford,

or that we'd like to do,

as well as whether we'd like a design for which we can estimate

all main effects or all the main effects,

as well as all the two-factor interactions or the full response surface model.

We have the option of generating parallel plots,

something we can use to help us select the right design.

I'll show you how that works. So I'll press okay,

I can put in the names of my factors as well as the high and low settings,

but I'm just going to leave it the way it is for now.

I've been given this table with 2,027 designs that satisfy our requirements.

Each one has five factors less than or equal to 35 runs,

and we can fit a full response surface model.

So how can we now select the best one?

Well, one thing we could use is the local data filter,

where we can select runs of a certain design of a certain run length

with or without center points, as well as our efficiencies.

The average or max variance of prediction and the powers for the intercept,

the main effects, and then the minimum and average powers

for the two-factor interactions in the square terms.

If we have a full response surface model.

Because we generated the parallel plots,

we also have the parallel plot here.

We can use all of this to then zero in

on designs that are the best among all the ones that we've chosen.

If the minimum power of the square terms

was something that was important to us, and I can narrow my search to 10 designs

rather than from among the 2,000 designs that were possible,

once I've done that, I can press this,

Get Summary Results script on the table, and then generate this table here

with the names of the designs, whether the design includes

a center point or not, the number of runs, as well as all of the metrics.

Let's see, I think I'll go ahead and just choose this one here.

I can press make design, and now I've been given this design a JMP.

One thing I'll add is that if you choose a design with the center point,

it adds a -0 at the end to indicate

that the center point has been added to that design.

I can now go ahead and add my response column,

save the table, and I'm ready to start conducting my experiment.

As Phil mentioned before,

Definitive Screening Designs are a subset of OMARS Designs.

Of course, there are many other

Orthogonal Minimally Aliased Designs that are not Definitive Screening Designs.

I'll show you an example here that uses six factors and a maximum of 20 runs.

In this case, we only have eight designs that meet this criteria.

I'm just going to go ahead and select all of them and press Get Summary Results.

Now, this 13- run design here with the center point

is actually the Definitive Screening Design for six factors.

You can see that this design is in every way

except for the power for the intercept, better than this 15- run O MARS Design,

which is not a Definitive Screening Design.

But I'm going to go ahead and select both of these

so that I can compare the designs.

When I do that, it'll open both tables as well as this compare designs platform.

Scrolling down to the color map on correlations,

I can see that the Definitive Screening Design,

which is this one here, looks as I would expect it to.

Of course, the O MARS Design is also orthogonal for the main effects.

That's what defines it as an O MAR Design.

But you'll also notice that

this one happens to be orthogonal

for many of the higher order effects as well.

If I were to try to fit the full response surface model

to add those terms to this model,

of course, I won't be able to add all of them,

but I'm able to add one additional term or to fit one additional term.

Using my O MARS Design,

then I would be using the 13- run Definitive Screening Design.

If I tried to do that ,

so now you'll notice that the powers for the intercept, the main effects,

and the quadratics are all higher for the O MARS Design.

They are lower than the Definitive Screening Design

for the interaction terms.

Looking at the fraction of the design space plot,

you'll see that the OMARS Design has a higher maximum prediction variance,

but is lower than the Definitive Screening Design over

more than 80 percent of the design space.

Interestingly, the Definitive Screening Design platform

doesn't have the ability to generate 15- run six factor designs.

We can generate 13 or 17runs.

If we can't afford 17-runs, but can afford 15,

this provides us perhaps an option that may be suited to us

that we'd like to consider or explore further.

Once again, thank you all for your attention.

At this point, I'd like to turn things back over to Phil.

Thanks, Hadley.

Just to summarize what you've seen there, what we've shown you,

hopefully, you've seen how these Orthogonal Minimally Aliased

Response Surface designs

can bridge that gap between the small, efficient Definitive Screening Designs

and large high- powered traditional response surface method designs.

You've seen how there's more flexibility.

There are Orthogonal Minimally Aliased Designs with three levels

for different numbers of runs now.

If a Definitive Screen Design doesn't meet your needs

or a traditional Response Surface Method design doesn't meet your needs,

you should now be able to explore these OMARS Designs.

Exploring those designs is now made easier to use as a JMP user

with the add-in that Hadley has created for you.

We'll obviously post links to all of these things in the article

in the community,

and that's a great place to let us know if you've got any questions as well.

Thanks very much for your attention.

Comments

What is OMARS DOE power for estimating the 2nd order interaction effects?

Phil_Kay

Hi @frankderuyck ,

I suggest that you take a look at the addin (attached) and generate some OMARS designs for yourself so that you can evaluate the power for 2-way interactions. The power will depend on the specific OMARS design, and your estimates of effect size and RMSE, of course.

In the example I showed in the presentation it is not possible to estimate all interactions with the DSD. But I have compared powers for some of the interactions (using default effect size and RMSE). It is not surprising that 31-run design (orange) has more power than the 17-run design (blue):

 

Phil_Kay_3-1648120982047.png

 

 

 

 

 

Victor_G

@Phil_Kay : I just got my answer about loss of orthogonality when augmenting a DSD or OMARS designs by watching the presentation of Bradley Jones about the use of Fit DSD on foldover designs : A Surprising Use of the Fit Definitive Screening Platform (2022-EU-45MP-1066) In order to augment these designs (DSD or OMARS) but preserving the nice orthogonality between main effects, and the orthogonality between main effects and 2-factors interactions, the trick may be to set the optimality criterion to "Make Alias Optimal Design". In this way, the special orthogonality properties of DSD or OMARS design can be kept, and augmentation will only serve to reduce the correlation between 2 FI and/or between quadratic effects :)

Example with correlation maps of a 18-runs DSD and the augmented Alias-optimal version with 26 runs in screenshot :

DSD-18runs-3continuous+2categorical_factors.pngAugmentedDSD-26runs-Alias_optimality-3continuous+2categorical_factors.png

Phil_Kay

Hi @Victor_G .Thanks. That makes sense. The Alias-Optimal criterion should do a good job of minimising correlation, as the name suggests. I am not sure that it will always create an OMARS design though. I have not properly explored this.

(Side note: the 18-run design that you created looks like the DSD for 3 continuous and 2 categorical factors. This is not strictly speaking an OMARS as the X4 and X5 main effects are not orthogonal to the other main effects. Hence the augmented design is also not an OMARS)

Victor_G

Hi @Phil_Kay
You're right, I haven't checked if augmenting a DSD with Alias-Optimal criterion will create an OMARS design, but I don't think it will be the case (perhaps sometimes by "luck", but clearly not by default). I will also check if some OMARS design may have an "internal" DSD in their design points, it might be a good solution for augmentation. 
You guess correctly, example was coming from a DSD with 3 continuous and 2 categorical variables. So not an OMARS design, but it was more to show how to keep the orthogonality properties when augmenting a DSD than using OMARS (or augmenting a DoE to an OMARS design). 

Phil_Kay

Sounds good, @Victor_G .

Thanks,

Phil

Here is the link to the original paper by Ares and Goos: https://www.tandfonline.com/doi/full/10.1080/00401706.2018.1549103

Victor_G

@HadleyMyers : Thanks Hadley !
A very instructive presentation of the OMARS designs has also been done by Ares & Goos, and showing a comparison between a DSD and an OMARS design, both with 22-runs and 5 factors. Here is the link : https://falltechnicalconference.org/wp-content/uploads/2017/11/webinar-FTC-Peter-Goos-OMARS-designs....
And here is their recorded presentation at the annual Fall Technical Conference 2020http://www.experimental-design.eu/videos/omars-designs/ 

Marco_

Hi @Phil_Kay 

Thanks for the instructive presentation and the attached files in this post.

Despite the orthogionaility of main effects with higher order effects it is however difficult for me to see where a OMAR design is more benifical in comparison to an optimal design. Maybe I have overseen something and you could point that out?

Phil_Kay

Hi @Marco_

The orthogonality of main effects and the minimal correlation between all other effect pairs are very much the benefits of OMARS. It "does exactly what it says on the tin", as we say in the UK. If those properties are not beneficial for your case then you should look to other design approaches.

An optimal design will always be the best design - again, that is what it says on the tin. However, it all depends on how you define optimal. 

My personal view is that OMARS are academically interesting. I would certainly recommend using DSDs (a subset of OMARS as we say in the presentation) in the right situation, i.e. screening of many mostly continuous factors. I don't think there are a lot of other applications where you would use an OMARS design.

Article Tags