How to make D-optimal Design properly?

[#1]

Hello, 

I am now following the practice in a textbook but I cannot mimic the auther's work.

The book is Douglas C. Montgomery, Design and analysis of experiments (2020)

On page 445 ~ 448, Example 11.3, summary is as follow.

4 factors,

in exact 16 runs,

D-optimal 

Using JMP

The result is,

However, I cannot mimic this result since,

JMP>Custom DOE>

- 4 Continuous factors

- RSM

- 'D-optimality' 

- runs: 16

above 4 are my designated values, as per following figure

However, the result is not same to the textbook.

It always includes below decimal Level of the factor(s)., as follows.

Thus there is no identity between the textbook.

What point did I miss?

Please tell me how to get through.

[#2]

Hi @MikeKim,

As experimental points from optimal designs are generated through an algorithm (Coordinate-Exchange Algorithm) and based initially on random points, a part of randomness is present and may lead to different optimal designs, with similar performances and characteristics but different individual experimental values for the factors and runs.

For the "strange" values you can sometimes see (like 0.81 instead of 1, or -0.22 instead of 0), this is due to the design generation process based on initial random values. Modifying the number of starts and the time for each design generation may improve the global optimality of the design found, but you may still encounter such values. This is not a big problem and you can still "round up" these values to more conventional ones (for example, rounding 0.81 to 1), with only a possible little loss in optimality.

If you want to use the same Custom Design as another one, the best option is to set a random seed, so that initial random points are the same and the convergence will lead to the same design : Solved: Re: Is it possible to make DOE generation reproducible? - JMP User Community

For this example, if you do not have the seed used to generate this design, it may be hard to find the same exact optimal design, but you can try generate it multiple times, and using the options to augment Number of Starts and Design Search Time.

I hope this answer will help you;

[#3]

Thank you Victor.

My opinion is, actually, there must have been a way to fix the Level of factors to become Integer.

Since 2 of the designs (D and I optimal) shows all the Levels are Integer (-1, 0, 1), it could have not been generated by a random manner.

Moreover, the overall performance of the generated Designs in my setting are a lot deficient than the Author's.

Says, for example, Pred Var: 2.2 (mine) and 1.8 (author).

It actually makes no sense that considering the some Users want to the Level of integer, JMP is not equipped with that Option, so I suspect there must be a way to do that.

So Victor, would you please re-consider this problem? If you have a chance, please refer directly to the textbook for more information than below if needed.

[#4 · Accepted solution]

Hi @MikeKim,

Of course there is a way to inform JMP that you have discrete numerical factors, just set up your 4 factors as Discrete Numeric with three levels (-1, 0, 1) :

This will add a constraint during the design generation to only use the three possible levels values for each factor in the design :

But that doesn't change the fact that design generation for optimal design in JMP is "random-based", please see my previous post about the Coordinate Exchange Algorithm and the response from @Mark_Bailey for explanations and informations. So you might end up with a design having similar performances, but different runs.

The one I generated with the same model (main effects, interactions and quadratic effects), optimality criterion (D) and run size (16) has a similar prediction variance as the D-optimal design from the book (1,86 vs. 1,80 in the book for the specific setting of factors) :

Please find attached the D-Optimal design I created with the discrete numeric factors. 

Following the same factor settings and methodology, you should be able to generate a satisfactory I-optimal design, with similar performances and characteristics than the one in the textbook.

I hope this answer will help you,

[#5]

Although it is awared that designating the Factors be Discrete results the Design to become Integer based, and considering Auther's intention and the context of the problem are unlikely to designate the Factors as discrete, you have shown so much interest to my question, so I thank you.

At least two of the JMP experts showed that there is no actual mimic of the textbook, so there might be a mistake in the textbook or misunderstanding, or version of JMP is different to make different results.

Thank you for your kind responses.

[#6]

Glad that my answer has been helpful. Note that the Discrete Numeric factors "trick" can be used for the optimal design generation, but once the design obtained, you can revert factors modeling type to Numeric Continuous to match with the original context of the problem.

I really recommend you to try generating several designs with the approach I mentioned above (with discrete numeric factors), and read the ressources listed (and others available) about the Coordinates Exchange Algorithm used in the platform Custom Design.
Sorry to insist on this, the only way to mimic exactly the design from the book in one trial is to have the random seed used for the design generation, and use it in JMP as well (there was one topic I have listed in my previous answers dealing with this aspect). Fixing the random seed enables to start with the same initial random points and to converge (during the design generation) to the same final optimal design. If you don't have this information, it is highly likely that several optimal designs may be possible to create, with similar performances and characteristics, that correspond to your assumed model, factors specifications and constraints. In this use case, several optimal designs are possible, there is no single unique possible optimal design, so that's why I'm not getting the same design as in the textbook but another one with very close performances.

There is no mistakes on textbook or JMP side, the randomness behind the optimal design generation enables to converge towards multiple similar (in terms of performances, but different in terms of experimental runs) designs.

I hope this complementary answer will dismiss any misunderstanding,

[#7]

Also, remember that in many situations, there is more than one design that is equally optimal because it has the same value for the optimality criterion as another design. The design might look very different to you regarding the runs, but they are the same as evaluated by JMP.

Note that you can click Back at the bottom of the Custom Design platform and then click Make Design again to get another design unless there is only one solution.