ELI5: D/I/A Optimal designs

[#1]

Hi JMP Community,

ELI5 stands for Explain like I'm 5 years old. 

I need some support explaining the differences between D-Optimal, I-Optimal and A-Optimal designs to colleagues which do not have strong mathematical backgrounds and cannot easily intuit the definitions in the JMP manual (I don't have a strong mathematical background either!).

Is there a way to explain what each of the D-Optimal, I-Optimal and A-Optimal algorithms are trying to achieve, which might hint at the advantages and disadvantages of each design type? 

Please let me know if you need more context. I appreciate any support on this topic!

I hope everyone is staying safe.

- Donald

[#2]

I'll take a stab...and I'm certain I'll miss a key point or maybe oversimplify, but here goes:

All experimental designs have relatively weak and strong points and have all been created to solve certain types of practical problems. The rich part of DOE is there is no one design that works 'best', whatever your criteria for 'best' might be, for all types of problems. Hence there is a broad category of designs that have been created for the experimental investigation scenario where one of the following conditions is present and can't be ignored:

1. The number of runs needs to be explicitly defined.

2. There are constraints on the factor settings within the experimental design itself.

3. There is a specific model the for which the investigator would like to detect active effects.

And you want to accomplish this all as efficiently as possible...and my measure of efficiency is "provides the required information for the least expenditure of resources".

When one or more of these conditions is present in the study, then generally speaking the most efficient family of designs to consider should be the family known as 'optimal experimental designs'...codified in JMP within the JMP Custom Design platform.

Within optimal designs there are many types...all named after something called an optimality criteria. Hence D-optimal, A-optimal, I-Optimal and so on. The way optimal designs work is once you articulate factors, levels, constraints, number of runs and your model...then a little black box goes into work and the design is created in such a way that for the optimality criteria you have chosen, is maximized. Now the question becomes, which optimality criteria should you choose...well there are two answers to this:

1. Let JMP figure it out for you! The way JMP is wired, once you define factors, constraints, model, number of runs, JMP will pick an optimality criteria that will have the best shot at providing the required information you are looking for. Think of this as the Big Red Easy button in JMP's DOE space. Is this 'best shot' a guarantee? Nope...but some pretty smart people at JMP and in DOE practice hit for some pretty high batting averages using these heuristics.

2. Force JMP to create a design for a specific optimality criteria...here's where it gets a little dicey...generally speaking you fit D-optimal designs when you are looking to estimate active effects...an I-optimal design when interested in fitting curvilinear responses, and an A-optimal design when your interest is minimizing average variance of the effects across the factor space...can be helpful when you really want to focus on estimating linear and quadratic effects relative each other.

Whew...that's about it for me...I'm sure I've left some stuff out...and maybe even gotten something a 'little' wrong or overgeneralized...but that's my story and I'm sticking to it.

I invite some of former colleagues and friends such as Mark Bailey and Statman to chime in.

[#3]

Peter,

Thank you for sharing so much information in a brief format.

I always continue to learn from these discussions. 

An amazing amount of knowledge and insights can be gained from the community of world class experts like yourself.

Stan

[#4]

You are too kind @stan_koprowski !

I still miss the days of sharing what I knew and learned 'oh so the hard way' as maybe @louv and I would have said?

[#5]

Peter has done all of the heavy lifting!

My "short" answers to the different types of designs are geared more around the mathematical side of it.

D-Optimal designs: Find the design that will minimize the variance of the parameter estimates of your model. These designs will typically put more runs at the edges of the design space.

I-Optimal designs: Finds the design that will minimize the variance of the predictions over the design space. These designs will typically put more runs in the interior of the design space than the D-optimal designs.

A-Optimal designs: These are more difficult to describe. I think of them as a blend between the D and I-optimal designs, but mathematically that is not true. An A-optimal design will minimize the sum of the variances of the parameter estimates. This is slightly different than the D-optimal criteria.

Ultimately, D, I, and A-optimal designs are all good at estimating parameters and good at prediction variances for the intended models and a sufficient number of runs. They are just truly optimized for the various criteria.

[#6]

I really don't have much to add to the excellent explanations given by Pete, Stan, Dan and Mark (the fantastic 4...).  My suggestion is for each situation, create multiple designs and evaluate them for their ability to provide the information you want compared to the resources required.  Prediction of expected results is a good idea.

The additional "element" I consider in selection criteria is NOISE. Unfortunately, having a focus on just the design structure and no strategy for noise leaves experiments that aren't useful in the real world and whose results can't be extrapolated.

[#7]

Thank you for everyone's fantastic replies. This is exactly what I was after.

[#8 · Accepted solution]

What is it good for? Another way to think about the optimality criteria is how they will help you with your goal.

D-optimality should provide you with the overall narrowest confidence interval estimates of parameter estimates. That result is useful when your goal is precise estimates of model parameters, or powerful tests of significant estimates. I might use a D-optimal design when I need to determine which factors or effects (terms) are significant. An extreme example of a case when D-optimal is a good choice is in a screening experiment.

I-optimality should provide you with the smallest confidence interval estimates of the response. That result is useful when your goal is precise estimates of the response, such as when you are optimizing factor settings to obtain the desired response.

A-optimality should also give you results like a D-optimal design by focusing on the quality of the parameter estimates (not the response estimates). They also offer you the opportunity to emphasize particular terms in your model by adjusting the weight applied to each term before summing the variances. You can adjust these weights in order to obtain a different set of power estimates for each term.