topic Re: Confirmation results: Redo mixture design or make a new one? in Discussions

Confirmation results: Redo mixture design or make a new one?

K_JMP — Tue, 06 May 2025 07:53:48 GMT

Hi JMP community,

I have conducted the following mixture design with constraints: MeOH + ACN + UPW = 0.6, and MeOH + ACN ≤ 0.3.

Would it be better to repeat this mixture design to confirm the results, or create a new one for validation?

Thanks a lot!

Kimberly

Details mixture design:

Version: Jmp Pro 18

Re: Confirmation results: Redo mixture design or make a new one?

Victor_G — Tue, 06 May 2025 10:39:14 GMT

Hi @K_JMP,

I'm not sure to fully understand your question.

Have you already run the design ? How reliable are the results, how accurate are the models ? I believe there might be a constrained ranges for factors missing in your description, as if I only use the one you mentioned with MeOH and ACN, I have a slightly different design space :

So MeOH ≥ 0,1 and ACN ≤ 0,15 seem to work based on your screenshot :

What would you like to validate : optimum or model ?

If you want to validate the optimum, I would use the model found and use the Prediction Profiler to find the optimum (or best compromise), and then run 3-5 times the optimum found, and check that measured values are included in the prediction intervals of the model for the optimum point (see Prediction Profiler Options to know how to display Prediction Intervals in Profiler).
If you want to validate the model, I would try to find a mix of different points covering different experimental zones with different desirabilities and prediction variance. A good option could be to use the Augment Designs platform and use the same assumed model (or a slightly more complex one, to avoid only "repeating" construction points from the original design) to generate new points (original points in blue, augmented points are green triangles):

You could then use these new points as validation points for your assumed model : fit/train the assumed model with the original points, and validate it with the new points. You can then look at various metrics (R², R² adjusted, RMSE, ...) on the validation set to check if the model seems relevant or may need some refining.

Hope these ideas may help you,

Re: Confirmation results: Redo mixture design or make a new one?

statman — Tue, 06 May 2025 13:02:51 GMT

I agree with Victor, I'm not sure what you are asking. If you have already decided a mixture design was the correct design and you completed it, then select the best space based on the mixture response surface and run the process (mixture designs are typically optimization designs meaning you have already determined these are the factors you want to model).

Re: Confirmation results: Redo mixture design or make a new one?

K_JMP — Fri, 16 May 2025 10:27:39 GMT

Hi @Victor_G & @statman

Thank you for the answers! My apologies as my message wasn't clear.

Let's give first some additional background information:

I have indeed included some other ranges as well:
- MeOH between 0.1 and 0.3
- ACN between 0 and 0.15
- UPW between 0.3 and 0.5
- And I have also set that the sum of the mixture: MeOH + ACN + UPW = 0.6
The design includes second-order interactions and one third-order interaction (MeOH × ACN × UPW) to capture the influential relationships between the components.
It is an I-optimal design, which also includes three center points, for a total of 12 runs.

What I actually want to do is to confirm the results (to make sure the results of the included points are trustable) and I want to explore the space some more (as I am scared I am missing some trends due to the limited points in the design space). At the end, I solely want to be certain that I selected an acceptable optimum (as I have different responses to evaluate and combine, leading to the final ratio's).

Based on both your inputs, I thought about the following:

I have run this design of 12 runs recently (results can be implemented as block 1) and I plan to rerun the current design twice again, assigning the new runs to separate blocks to account for variability across experiments. This would allow me to assess systematic effects and lower the standard deviation of the results. Consequently, when I use the Augment Design option, this leads to 36 runs.
Additionally, to explore the additional design space and potentially uncover nonlinear trends, I am considering augmenting the design again by adding Scheffé cubic points. These can perhaps serve as validation points as mentioned by @Victor_G. For example, this can lead to 44 runs with the additional points highlighted in red.

Remaining question/concern:

When performing the 2nd augmentation step (so adding the Scheffé cubic points), I definitely see an increase in the number of prediction variances (to 0.57) and also an increase in Fraction of design Space plot (as you can see below). It makes sense that this increases, however I am just not really certain if this is the smartest move to do for my purpose?

Thanks in advance!

Kind regards

Kimberly

Re: Confirmation results: Redo mixture design or make a new one?

Victor_G — Fri, 16 May 2025 13:39:24 GMT

Hi Kimberly,

I'm not sure about the last comment/question :

@K_JMP wrote:

Remaining question/concern:

When performing the 2nd augmentation step (so adding the Scheffé cubic points), I definitely see an increase in the number of prediction variances (to 0.57) and also an increase in Fraction of design Space plot (as you can see below). It makes sense that this increases, however I am just not really certain if this is the smartest move to do for my purpose?

You should use the platform Compare Designs to make sure the Prediction Variance Profile and Fraction of Design Space Plot are calculated on the same assumed model. If you compare these information on each "Evaluate Designs" designs' script, the comparison is not the same, as the original and replicated designs have the same assumed model, but not the augmented one, since Scheffe cubic terms were added.

When using the Compare Designs platform with the 3 designs, the comparison of the designs is done on the same assumed model, and the comparative results are completely logical:

You can clearly see a big decrease of prediction variance from the original I-Optimal design to the Replicated Design, and another decrease from the Replicated design to the Augmented design, which make totally sense : big decrease first thanks to replication, then thanks to adding points to better support the first initial model and validate it.

Hope this answer will help you,