topic Re: Unexpected change in controlled variable in DOE in Discussions

Unexpected change in controlled variable in DOE

Mathej01 — Fri, 01 Mar 2024 12:48:24 GMT

Hi,

I made a custom design to understand the impact of caoting temperature and speed on performance. I kept all the parameters same. However, unexpectedly I had to vary the nozzle air pressure for different runs as it was not possible to make the coatings at same air pressure. So, my question here is that how can i include the change in air pressure in the DOE. I cannot afford any more runs.

Is there any possibility to consider the different air pressure or should I just assume that it does not have any affect on final performance(which I am not sure).

Is my question clear?

Thanks in advance

Re: Unexpected change in controlled variable in DOE

Victor_G — Fri, 01 Mar 2024 14:15:53 GMT

Hi @Mathej01,

If nozzle air pressure is a new variable in your experimental setup, you can use the values and treat them as an Uncontrolled Factor : "An uncontrolled factor is one whose values cannot be controlled during production, but it is a factor that you want to include in the model. It is assumed that you can record the factor's value for each experimental run."

You can simply add a new column in your datatable with your nozzle air pressure values, and add the column properties "Design Role" (Uncontrolled), "Factor Changes" (Easy) and "Coding" (add the min and max values in the low and high values).

In your model, you can then add this new uncontrolled factor as a main effect to better assess its importance on the response(s) in your experimental setup.

I hope this answer will help you,

Re: Unexpected change in controlled variable in DOE

SamGardner — Fri, 01 Mar 2024 13:56:18 GMT

You can just edit the values for Air Pressure to be the actual values and then do the analysis. It may not be statistically optimal design anymore, but you may still get a useful model. If you know a bit about design evaluation metrics, you can also use the Evaluate Design tool or the Compare Designs tools (DOE > Design Diagnostics ) to see the impact that the changes in the factor levels have on the confounding in the design.

Re: Unexpected change in controlled variable in DOE

statman — Fri, 01 Mar 2024 15:25:23 GMT

Here are my questions and thoughts:

Do you have an actual value for the nozzle air pressure? Was the actual air pressure measured? How is the nozzle air pressure controlled (e.g., is the a knob you turn?)? If it is by some knob (or other way to control the valve), do you have the reading for that setting? Is it a continuous variable or categorical? Did it change for every treatment in the experiment, or did it just change with associated changes in coating temperature and speed? Be careful with this! If the changes correlate with certain treatment combinations, you will likely have multicollinearity.

While in reality, this variable is controllable, but was just not included in your planned experiment, you have the option of treating that variable as a covariate (usually this is a strategy for handling an uncontrolled variable that can be measured). When you create the model for the analysis of the experiment, you will have to write a mixed model (that is. fixed effects for the experimental factors and interactions and the covariate as a random variable). You introduce potential multicollinearity into the analysis. You will need to check for correlation between the covariate and any of the model terms (fixed effects). This can be done with correlation matrices (Analyze>Multivariate Methods>Multivariate and enter all of the model terms including the covariate) and/or with VIF's after running Fit Model (right click on the Parameter Estimates table>Columns>VIF). I suggest you write the model (for Fit Model analysis) with the covariate first and then the fixed effects. You should test the significance of the covariate with Sequential Tests (red triangle>Estimates>Sequential Tests).

Notes to self:

1. Be more thorough in identifying variables before you run your experiment. I suggest Process Mapping (https://www.tandfonline.com/doi/abs/10.1080/08982119908919275) the experiment before running the experiment to identify controllable and noise factors.

2. Before you run any experiment, predict the results you will get. One reason for this, is this allows you to think through the possible combinations to determine their reasonableness.

"To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of."

Sir Ronald Fisher

Re: Unexpected change in controlled variable in DOE

Mathej01 — Mon, 04 Mar 2024 11:48:06 GMT

Thanks @Victor_G . It was helpful. Even though it was unexpected, I have values for nozzle air pressure.

Also as a follow up question, can i add it as a noise instead of uncontrolled factor? Does it make sense ?

Re: Unexpected change in controlled variable in DOE

Mathej01 — Mon, 04 Mar 2024 11:49:31 GMT

Thanks @SamGardner . I am not able to compare the designs in this case.

Re: Unexpected change in controlled variable in DOE

SamGardner — Mon, 04 Mar 2024 12:08:19 GMT

To compare the original design the design "as it was executed", make a copy of the original design, modify the factor settings in the copy, and use Compare Designs to compare the original and the modified design.

Re: Unexpected change in controlled variable in DOE

Victor_G — Mon, 04 Mar 2024 16:05:56 GMT

If I understood well the topic, it's not a question of not having the correct values of nozzle air pressure in the design, it's a question of not having included it in the design as a factor.
It was supposed to be fixed at a constant value, but this was not possible experimentally, so @Mathej01 did record the actual values to take them into account in the analysis.

@Mathej01 The design role "Noise" is only available once your design has been created and the datatable is generated (it's not part of the main factors types in the design creation for Custom Design). Noise factors are variables that are difficult or expensive to control in production. However, you must be able to control noise factors during the experiment.
You can indeed change the "Uncontrolled" type in the "Noise" type if you're interested in a robust optimization regarding the nozzle air pressure values, meaning having the best and most robust desirability in the presence of a noise factor (= control the response variation due to the noise factor). You can find an example of an optimization with a noise factor here : Example of a Noise Factor in the Prediction Profiler

Hope this will help you,

Re: Unexpected change in controlled variable in DOE

Julianveda — Tue, 28 May 2024 08:19:17 GMT

Hello @statman ,

I am confuse with the definition of a covariate. It is defined here as an "uncontrollable variable that can be measured". Similar definitions are found in these posts:

Exploring data with ANCOVA - JMP User Community

What is a covariate in design of experiments? (jmp.com)

My question is, what would be the difference of "covariate" vs "uncontrolled" in DoE ?

I used in the past covariate as an uncontrollable factor (I was using other software with this terminology). However, in jmp these are 2 different things yet the definition are close enough to create confusion at least for non statisticians.

Could you please help in clarifying the differences ?

thanks,

Julian

Re: Unexpected change in controlled variable in DOE

Victor_G — Tue, 28 May 2024 15:24:12 GMT

Hi @Julianveda,

Covariate is a variable that you want to account for in the model, cannot control it, but you know the values in advance (ahead of the experiment) and you hope that with a good covariates space representativeness and a "good" model, you might be able to understand the link between the covariates and the response(s), and use the model to select the right settings/values of the covariate factors to optimize your process/experiment.
You can think about physico-chemical properties of raw materials : depending on the chemical structures and properties you might not have the same input values all the time. But these values can be measured or known before doing the experiments. If you take these properties into consideration, select a representative subset of chemicals, use them in a DoE as covariates and analyze the results with a sufficiently "good" model, you might better understand how these molecular properties might affect the response(s). And so you might be able to select next time the chemicals with the right properties to optimize your response(s).

Uncontrolled is a variable that you want to account in the model, but you only know the values during the experiment, at each run, and it also can't be controlled (as the name suggests).
You can think about temperature or pressure during the experiment: you don't know in advance how these factors might change during your experiments, but you can record the values for each run and account for this variability in the response(s). But you might not be able to set up or fix these uncontrolled factors at specific values later in order to optimize your process/experiment.

Hope this clarify,

Re: Unexpected change in controlled variable in DOE

statman — Tue, 28 May 2024 16:29:14 GMT

I'm not sure I agree with the distinctions made. A covariate is a measurable noise variable (a variable you are not willing to control for what ever reason). Since you can put a value in for each treatment, this random variable can be assigned in the model typically with 1 DF) and thus reduces the estimate of the MSE. You will be writing a mixed model.

Now there are issues with using covariates. For example, you are limited as to how many covariates you can account for. What value to you use for the covariate? Let's say the covariate is a chemical property as Victor alludes to. Realize the chemical property varies and the value you use may not be exactly correct. You have additional measurement errors measuring the covariate....

Re: Unexpected change in controlled variable in DOE

Victor_G — Tue, 28 May 2024 16:49:55 GMT

I may be wrong, but the distinction between Uncontrolled variable and covariate relates to two aspects for me:

Time-dependancy : covariates can be measured or calculated in advance, before running the experiments, whereas Uncontrolled variable are measured only during the experiments.
Use of this variable : covariates can be used to select/filter the most promising conditions to optimize an experiment, whereas an Uncontrolled factor is just a source of variability, but to no interest for the practitioner.

In my example, temperature and pression may also have some measurement errors like any measured input factors, so I don't think this could make a distinction between covariates and uncontrolled factors. Besides, I was referring the molecular properties used as covariates as the properties you can calculate (also called "molecular descriptors" so no measurement errors involved, but still some inaccuracies/deviations are possible).

Re: Unexpected change in controlled variable in DOE

statman — Tue, 28 May 2024 18:19:44 GMT

I'm not sure about right and wrong, but I don't think your time dependency is required to use covariates. Only that you can take a measure of the covariate for each treatment. Also I don't think I understand your Use of this Variable statement. The covariate is a random variable in an otherwise fixed effects model. Thus you have a mixed model. Accounting for the covariate reduces the MSE estimate (if this was not accounted for, the MSE would include the effect of the covariate). One could argue this increases the precision of the experiment. As a random variable, I typically start with adding the first order effect, but there may be additional effects you can estimate (interactions and non-linear). If the covariate is indeed significant, the user may be able to input the covariate value in the model and solve for remaining significant factors (elect levels) in the model to improve the results.

I am always interested in determining causal relationships. It is our hope (wish) that we can develop a useful model using factors that we are willing to manage, but that is not required of nature. It may be in the noise which is extremely useful to the practitioner. Knowing the significant variation is from noise leads one to broaden their investigation. One may find they are willing to manage factors they previously did not or one may desire to become robust to the noise both choices are important to the practitioner.

Re: Unexpected change in controlled variable in DOE

Julianveda — Wed, 05 Jun 2024 06:39:40 GMT

Thank you @Victor_G for your answer.

Just in case I am also exchanging about this subject in this thread: Solved: Define a covariate - JMP User Community

I explained there that I am confused because I have a variable (powder humidity) that I know in advance of my experiments and that it is in a certain way uncontrolled. I was confused because it is said that covariates are variables that are know in advance (like in my case). However, my variable (powder humidity) does not really co-varies with the change in the factor levels from one treatment to the next (as explained by @Mark_Bailey in the the other thread).

I am therefore more inclined to treat my variable as an uncontrolled one, but I am not completely sure of making the right choice.

regards,

Julian

Re: Unexpected change in controlled variable in DOE

Julianveda — Thu, 27 Jun 2024 06:53:57 GMT

Thank you @Victor_G and @statman for all your views on the subject. They contribute to better understanding the topic. Still, I see that sometimes the line between Covariate and Uncontrolled variables is very fine.

Re: Unexpected change in controlled variable in DOE

statman — Thu, 27 Jun 2024 14:12:55 GMT

You are confusing the nature of the variable (e.g., uncontrolled or noise) with how it can be handled in an experiment. One way to handle an uncontrolled variable that can be measured is to treat it as a covariate. There are other ways to handle noise in an experiment.