Unexpected change in controlled variable in DOE

[#1]

Hi,

I made a custom design to understand the impact of caoting temperature and speed on performance. I kept all the parameters same. However, unexpectedly I had to vary the nozzle air pressure for different runs as it was not possible to make the coatings at same air pressure. So, my question here is that how can i include the change in air pressure in the DOE. I cannot afford any more runs.

Is there any possibility to consider the different air pressure or should I just assume that it does not have any affect on final performance(which I am not sure).

Is my question clear?

Thanks in advance

[#2 · Accepted solution]

Hi @Mathej01,

If nozzle air pressure is a new variable in your experimental setup, you can use the values and treat them as an Uncontrolled Factor : "An uncontrolled factor is one whose values cannot be controlled during production, but it is a factor that you want to include in the model. It is assumed that you can record the factor's value for each experimental run."

You can simply add a new column in your datatable with your nozzle air pressure values, and add the column properties "Design Role" (Uncontrolled), "Factor Changes" (Easy) and "Coding" (add the min and max values in the low and high values). 

In your model, you can then add this new uncontrolled factor as a main effect to better assess its importance on the response(s) in your experimental setup.

I hope this answer will help you,

[#3]

Thanks @Victor_G . It was helpful. Even though it was unexpected, I have values for nozzle air pressure. 

Also as a follow up question, can i add it as a noise instead of uncontrolled factor? Does it make sense ?

[#4]

You can just edit the values for Air Pressure to be the actual values and then do the analysis.  It may not be statistically optimal design anymore, but you may still get a useful model.  If you know a bit about design evaluation metrics, you can also use the Evaluate Design tool or the Compare Designs tools (DOE > Design Diagnostics ) to see the impact that the changes in the factor levels have on the confounding in the design.  

[#5]

Thanks @SamGardner . I am not able to compare the designs in this case. 

[#6]

To compare the original design the design "as it was executed", make a copy of the original design, modify the factor settings in the copy, and use Compare Designs to compare the original and the modified design.  

[#7]

If I understood well the topic, it's not a question of not having the correct values of nozzle air pressure in the design, it's a question of not having included it in the design as a factor.
It was supposed to be fixed at a constant value, but this was not possible experimentally, so @Mathej01 did record the actual values to take them into account in the analysis.

@Mathej01 The design role "Noise" is only available once your design has been created and the datatable is generated (it's not part of the main factors types in the design creation for Custom Design).  Noise factors are variables that are difficult or expensive to control in production. However, you must be able to control noise factors during the experiment.
You can indeed change the "Uncontrolled" type in the "Noise" type if you're interested in a robust optimization regarding the nozzle air pressure values, meaning having the best and most robust desirability in the presence of a noise factor (= control the response variation due to the noise factor). You can find an example of an optimization with a noise factor here : Example of a Noise Factor in the Prediction Profiler 

Hope this will help you,

[#8 · Accepted solution]

Here are my questions and thoughts:

Do you have an actual value for the nozzle air pressure? Was the actual air pressure measured?  How is the nozzle air pressure controlled (e.g., is the a knob you turn?)? If it is by some knob (or other way to control the valve), do you have the reading for that setting?  Is it a continuous variable or categorical?  Did it change for every treatment in the experiment, or did it just change with associated changes in coating temperature and speed?  Be careful with this!  If the changes correlate with certain treatment combinations, you will likely have multicollinearity.

While in reality, this variable is controllable, but was just not included in your planned experiment, you have the option of treating that variable as a covariate (usually this is a strategy for handling an uncontrolled variable that can be measured).  When you create the model for the analysis of the experiment, you will have to write a mixed model (that is. fixed effects for the experimental factors and interactions and the covariate as a random variable).  You introduce potential multicollinearity into the analysis.  You will need to check for correlation between the covariate and any of the model terms (fixed effects).  This can be done with correlation matrices (Analyze>Multivariate Methods>Multivariate and enter all of the model terms including the covariate) and/or with VIF's after running Fit Model (right click on the Parameter Estimates table>Columns>VIF).  I suggest you write the model (for Fit Model analysis) with the covariate first and then the fixed effects.  You should test the significance of the covariate with Sequential Tests (red triangle>Estimates>Sequential Tests).

Notes to self:

1. Be more thorough in identifying variables before you run your experiment.  I suggest Process Mapping (https://www.tandfonline.com/doi/abs/10.1080/08982119908919275) the experiment before running the experiment to identify controllable and noise factors.

2. Before you run any experiment, predict the results you will get.  One reason for this, is this allows you to think through the possible combinations to determine their reasonableness.

"To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of."

 Sir Ronald Fisher

[#9]

Hello @statman ,

I am confuse with the definition of a covariate. It is defined here as an "uncontrollable variable that can be measured". Similar definitions are found in these posts:

Exploring data with ANCOVA - JMP User Community

What is a covariate in design of experiments? (jmp.com)

My question is, what would be the difference of "covariate" vs "uncontrolled" in DoE ?

I used in  the past covariate as an uncontrollable factor (I was using other software with this terminology). However, in jmp these are 2 different things yet the definition are close enough to create confusion at least for non statisticians.

Could you please help in clarifying the differences ?

thanks,

Julian

[#10]

Hi @Julianveda,

Covariate is a variable that you want to account for in the model, cannot control it, but you know the values in advance (ahead of the experiment) and you hope that with a good covariates space representativeness and a "good" model, you might be able to understand the link between the covariates and the response(s), and use the model to select the right settings/values of the covariate factors to optimize your process/experiment.
You can think about physico-chemical properties of raw materials : depending on the chemical structures and properties you might not have the same input values all the time. But these values can be measured or known before doing the experiments. If you take these properties into consideration, select a representative subset of chemicals, use them in a DoE as covariates and analyze the results with a sufficiently "good" model, you might better understand how these molecular properties might affect the response(s). And so you might be able to select next time the chemicals with the right properties to optimize your response(s).

Uncontrolled is a variable that you want to account in the model, but you only know the values during the experiment, at each run, and it also can't be controlled (as the name suggests). 
You can think about temperature or pressure during the experiment: you don't know in advance how these factors might change during your experiments, but you can record the values for each run and account for this variability in the response(s). But you might not be able to set up or fix these uncontrolled factors at specific values later in order to optimize your process/experiment.

Hope this clarify,