topic Re: Mixture DOE with covariate batch effects in Discussions

Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 06:59:08 GMT

I have to generate a DOE with 7 mixture effects and 6 continuous covariate factors. The covariate factors are measurements of raw material batches, only 20 bathes are available. Mixture and quadratic covariariate effects as well as mixture -covariate interactions need to be estimate using a Kowalski with only 2nd order interactions. The number of runs exceeds #batches so repeated sampling from each batch is necessary. Batch effect is uncontrolled and random. How to set up this DOE? I tried to make a split plot with covariates hard to change so whole plot can be assigned as random however when making the DOE it fails to converge? Do I need to generate a large covariate table with multiple repetitions of the 20 batches?.. Thanks a lot for advice.

Re: Mixture DOE with covariate batch effects

Victor_G — Mon, 29 Jan 2024 15:18:24 GMT

Hi @frankderuyck,

I'm not sure to have understood the link between your covariate factors and the mixture factors.

Are the 6 covariate factors linked to one batch type (row 1: properties 1, 2, ... 6 correspond to first batch) ?
Are the batches related to one mixture factor, or several ones ?

What I would like to understand is the link between batches and mixture(s) factor(s) (to better understand the assumed model).

If you have mixture factors A, B, C, ... D, are the batch related only to one factor (A1, A2, A3, ...), or to several factors (A1, A2, B1, B2, B3, C1, C2, C3, ...) ?

Would you like all batches to be tested ? Or perhaps based on the covariate table for the 20 batches, a representative subset (based on covariates values) could be identified and used in the design generation?

Concerning the model, you may expect some redundancies when specifying terms depending on the assumed model. For example, as sum of mixture factors is equal to 1, main effects for non-mixture factors and interactions between non-mixture factors and mixture factors can't be present independently in the design as they are aliased (but JMP will inform you and deleted these terms if you have these redundancies):

What is your experimental budget ?

If you can provide more infos/screenshots about your earlier tests, it could help frame the topic and better help you.

Initiating the discussion,

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 15:50:37 GMT

Hi Victor, the covariates are linked to one mixture effect say X1 for which 20 batches of raw material are available. Each batch raw material for the mixture factor X1 is characterized by a set of 6 properties which have been measured. We want to estimate the main, quadratic effect of the raw material properties of mixture effect X1and also the interaction effects with the 7 mixture effects. This will require more runs than the 20 available covariate rows/batches. Guess we will have to specify a random batch effect?

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 16:03:42 GMT

In aatchment an interesting publication related to case study, guess this i the approach to follow like in the propylene case. Looks like a split plot is required?

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 16:13:48 GMT

I don't understand how to design an experiment with repeated measurements from a limited set of batches, each batch being characterized by several covariate factors. The batch effect can't be controlled so it is random.

Re: Mixture DOE with covariate batch effects

statman — Mon, 29 Jan 2024 17:09:40 GMT

Here are my thoughts: Admittedly, I do not understand your situation sufficiently to provide specific advice.

I believe a sequential approach would be more efficient than one BIG experiment.

I'm not sure I understand the use of the term covariate for your example, nor what you would actually do to your process if you have covariate by mixture factor interactions (other than you can say your process is not robust). It is a challenge to account for 1 covariate in a typical experiment. Accounting for 6 is likely not very efficient or effective. Repeated sampling from the same batch does not increase the degrees of freedom associated with an experiment whose experimental unit is defined as a batch.

I would start with directed sampling to understand the batch variation components. It seems you have at least 3 components of variation: Measurement, Within batch and Batch to batch. You should want to understand the relative size and the consistency of these components before planning your experiment. Apparently you have 6 measures (response variables or Y's) of the batch that provide insight to the batch variation? Do these correlate? Do you want to understand why there is within and between batches variation? How confident are you in measurement systems?

Mixture designs are optimization designs. They are not really intended for screening. I think you are in need of screening before optimization.

Re: Mixture DOE with covariate batch effects

Victor_G — Mon, 29 Jan 2024 20:06:21 GMT

Hi @frankderuyck,

From what I understand, you're interested into variability from batch to batch as a random effect, not a fixed effect.
So I would not complicate the topic, and instead use the 6 covariates to select the most "dissimilar" batches to ensure representativity of the batch variability, and only use these representative selected batches as blocks in the mixture experiments.

So to summarize, here are the steps I would do (if I understood correctly your goal) :

From the batches and covariates tables, create a D-Optimal Custom Design using the 6 covariates as the only factors, specify only main effects in the model, and use as few as possible experiments, so that the selection covers the batches experimental variability. Depending on the values in each covariate factor, you may end up with 7 (or more) selected batches selected (k: number of batches).
Create the Mixture design with the 7 mixture factors and a block factor, so that the total number of experiments is divided into the k+1 (at least) batch blocks. You won't have to specify in the model interactions between non-mixture and mixture effects or quadratic effects of non-mixture factors, since the only non-mixture factor you have with this setting is the random batch effect.
Once the table is generated, you can add the covariates corresponding to each batch block. Depending on the number of runs and number of whole plots, you may have some batch replicates.
In the analysis, you can analyze the results using a mixture mixed model, taking into account fixed mixture effects and random batch effect.
When you have estimated the BLUP (Best Linear Unbiased Predictor) attributed to each batch random effect, it may be possible to analyze these estimates changes based on covariates.

Would this process make sense to you ?

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 21:01:27 GMT

Hi Victor, this makes lot of sense to me, very thoughtful this all-in-one sequential DOE combination! Is this also the approach used in the paper I sent?

Re: Mixture DOE with covariate batch effects

statman — Mon, 29 Jan 2024 21:07:42 GMT

This makes no sense to me, so perhaps you can explain your terminology?

"each batch being characterized by several covariate factors". Why are you using the word covariate here? I typically think of covariates as independent factors that you cannot directly control (i.e., noise), but can be measured. Covariates are one of the many ways to handle noise in a designed experiment. Because they are assigned in the model as a random effect, they are accounted for and their influence on statistical tests is minimized (increasing the precision of the design).

I assume you have no ability to control the process making batches. Is this a supplied material? Are these factors input variables with respect to the batch "quality"? How do you know the relationship between the factors and the measures of the batch "quality"? Can you quantify the batch quality in a meaningful way (without using all 6 input factors)? A single measure of the batch quality may be useful as a covariate in an experiment.

Re: Mixture DOE with covariate batch effects

statman — Mon, 29 Jan 2024 21:10:29 GMT

I must be speaking a different language...

"all-in-one sequential DOE combination" is an oxymoron.

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 21:14:13 GMT

I found a similar mixture experiment by Peter Goos in literature "I-optimal design of split-plot mixture-process variable experiments: A case study on potato crisps" see attachment. A Kowalski model is used to estimate all main, interaction and quadratic effects including the covariates using 40 batches of a mixure factor. Giant experiment with 256 runs; I wonder how he fixed this DOE?

Re: Mixture DOE with covariate batch effects

frankderuyck — Mon, 29 Jan 2024 22:03:28 GMT

don't take it too literally just figure of speech

Re: Mixture DOE with covariate batch effects

Victor_G — Wed, 31 Jan 2024 14:14:18 GMT

Hi @frankderuyck,

I finally had the chance to read the paper and try to reproduce the same design (or at least having a similar design using the same methodology), following these steps:

Start by creating a table with 40 batches and 6 continuous numerical properties measured on these batches (file "Batch_Covariate_table"). These 6 properties will be used as covariates in the design.
Use the Custom Design platform, enter the mixture factors and ranges, add the covariate factors from the previous table and change their "Changes" property as "Hard" as mentioned in the paper to have a split-plot design structure type.
Change the optimality criterion to "I-Optimal" and specify the number of starts (10 instead of 1000 in the publication to save some time, but careful if you want to generate it again, design generation lasted approximately 1h on my computer (around 5min/random start) !).

Enter the linear constraints as mentioned in the paper :

{-0.6 * :Potato flakes + 0.4 * :Wheat starch + 0.4 * :Parboiled rice flour + 0.4 *
:Extruded rice flour + 0.4 * :Corn flour <= 0, -0.3 * :Potato flakes + 0.7 *
:Wheat starch + -0.3 * :Parboiled rice flour + -0.3 * :Extruded rice flour + -0.3 *
:Corn flour <= 0, -0.5 * :Potato flakes + -0.5 * :Wheat starch + 0.5 *
:Parboiled rice flour + -0.5 * :Extruded rice flour + -0.5 * :Corn flour <= 0, -0.5
 * :Potato flakes + -0.5 * :Wheat starch + -0.5 * :Parboiled rice flour + 0.5 *
:Extruded rice flour + -0.5 * :Corn flour <= 0, -0.5 * :Potato flakes + -0.5 *
:Wheat starch + -0.5 * :Parboiled rice flour + -0.5 * :Extruded rice flour + 0.5 *
:Corn flour <= 0}

Specify the model, with mixture main effects, 2-mixture factors interactions, quadratic non-mixture effects, and 2-non-mixture factors interactions (estimability set as "If Possible" for these last terms).
Set the number of whole plots to 40 (same number as the number of batches/covariates runs available) and number of runs to 256.
Make design !

As I wasn't able to had access tu supplementary materials, I'm not sure this is the same design they obtained. If I have taken into accounts all their requirements and specifications of the design, it should however be similar to the one they obtained. If you have access to the 256-runs table (and supplementary materials), you can compare the designs.
I would be interested to have it as well to check I didn't forget something when setting up the DoE :)

From what I understand in the design, the goal with the covariates here is to determine fixed effects of batch properties on the responses (fixed effects terms), as well as the influence of batch variability on the responses variance (whole plot random effect, and consideration of the 40 batches as a representative sample from a larger population).
Interesting design !

I hope this complementary answer will help you,

Re: Mixture DOE with covariate batch effects

frankderuyck — Wed, 31 Jan 2024 20:06:47 GMT

Hi Victor, great job, wish I coud give 10 Kudo's! I have no acces to the original DOE but I'm convinced it will be similar.

Indeed goal is to find interaction effects between raw material properties and mixture effects so robust settings might be possible or raw materials may have to be more controlled.

One remark on this paper: 256 runs is a lot.. I wonder if it is not possible with less runs and what minimum #runs are required to still achieve a useful and reliable model.

Thanks a lot for your great help!!

Frank

Re: Mixture DOE with covariate batch effects

Victor_G — Thu, 01 Feb 2024 07:43:11 GMT

Hi Frank,

I'm glad this design could work for you. I would still recommend some caution about the design generated, as in the absence of the original design from the paper, I'm not able to compare it or see if I missed something during its generation.

Regarding your concern about the high number of runs of this design, I see two options (that you can combine):

Reduce the number of batches in the design (to reduce the number of whole plots) : as described previously, you could start on your batch table by creating a small D-Optimal custom design on covariates (with only main effects in the model), in order to only keep the 7-10 most dissimilar/representative batches. This could heavily reduce the required number of runs, but the 2-factors non-mixture interactions coefficients as well as quadratic non-mixture coefficients may likely be less precisely estimated.
Simplify the model and/or set some higher order model terms estimability to "If Possible" : This could save some runs as well, but at the cost of a more simple model that may not accurately describe the system.

You can still calculate the required minimum of runs based on the number of factors (and terms in the assumed model) in your project to estimate if you should simplify your problem (remove some factors, do a screening first, etc...) or if the required number of runs is acceptable.

Hope this complementary answer may help you,

Best,

Re: Mixture DOE with covariate batch effects

frankderuyck — Thu, 01 Feb 2024 09:12:44 GMT

Hi Victor,

I agree, better use a sequential approach with a limited # batches based on a covariate selection DOE and "if possible" (a powerful) screening of higher order mixture effects. One can reserve a number of batches as a test set for validation of the model that can be augmented to a small space filling or special Scheffé mixture DOE.

Kind regards! Frank

Re: Mixture DOE with covariate batch effects

frankderuyck — Thu, 01 Feb 2024 11:32:40 GMT

Study of the raw material variance is difficult as batches are supplied by external vendors. I agree that - a discussed above - a pre-selection is recomended using a covariatie DOE that will yield the best selection.

Re: Mixture DOE with covariate batch effects

frankderuyck — Fri, 02 Feb 2024 08:54:20 GMT

I still have one question: on page 4 of the patato crisp paper Peter assigns a random block effect to the 40 batches?

However the batches are correlated with fixed hard to change covariate effects so I assume that when using a mixed model analysis assigning batch as random REML will not estimate the covariate effects?

Re: Mixture DOE with covariate batch effects

Victor_G — Fri, 02 Feb 2024 20:06:42 GMT

Hi @frankderuyck,

Yes, a random block effect is assigned due to the "hard-to-change" covariate factors which create a split-plot design structure type.

However, using a mixed model analysis does enable to estimate quadratic covariate effects, as well as 2-factors covariate interactions. You can try it using the datatable provided and creating a random response.
The "hard-to-change" covariate factors create a split-plot structure, which imply a restriction on randomization, but you're still able to estimate the assumed covariate terms estimates, at a lower precision than "easy-to-change" factors would have enabled.

Hope this answer will clarify,