How do I analyse factorial design

[#1]

Hi,

I would like some assistance. Recently I ran a 2^4 factorial. My response is some texture properties of my extrudates. However, during the experiments, there were some conditions in which I was unable to obtain extrudates and hence not able to get any texture data. Hence I would like some advice on how to analyze such designs with missing data points.

Thanks!

[#2 · Accepted solution]

Generally you can still analyze the experiment in the same manner as if you had all the responses. However, some of the model terms will be inestimable. Which terms depends on which treatment combinations are missing. I would also use the Evaluate Design platform excluding the rows that are 'missing' compared to the full complete design to see how much power you've lost and correlation of estimates plus other relevant design characteristics.

[#3]

Agree with P_Bartell to assess the design impact. You may be able to still fit the model. But you need to be very careful interpreting that model, especially around the failure region. 

Why the data are missing is critical. "I was unable to obtain extrudates"

Ask why this happened and whether it could be related to the factor settings used. For example, are you going to a corner of the space where the process goes over a cliff and fails? Sort the data table, what is in common with the failures? Is there a suitable upper (lower) bound on the response you could use as a surrogate in the data to explore the failure space? Also consider analyzing a binary 0/1 failure response. This is to give you insight on what causes the failure that you then confirm with follow-up experiments.

It is important to keep in mind that the "missing data" might be the most important data you collected. Do not discard it when interpreting the results. 

Consider augmentation. You might augment toward the failure corner along a vector to find the edge of failure first then augment the experiment using a constrained region, for example.

Does this typically happen in these types of experiments? If so, in future experiments, beefing up the design such as higher resolution and/or using larger models can make the design more robust to losses. Or performing some preliminary runs to map out the failure space may help. 

[#4]

I won't discuss the topics brought by previous responses, as they are perfectly clear and I couldn't agree more.
About design augmentation in the failure area, you could perhaps use a Space-Filling approach, to have more flexibility about the number of points used in the augmentation and more flexibility in the modeling of this area. It could perhaps enable to map more precisely the boundary "experimental feasibility" region.

About preliminary runs mentioned by @datadad2 , as you're using factorial designs you may use scoping designs to assess feasibility of the experimental runs at high, medium and low conditions of your experimental space : https://prismtc.co.uk/resources/blogs-and-articles/scoping-designs
It can help avoid failure in runs, assess reproducibility and noise, and might enable to detect curvature very early in your project.

[#5]

Echoing agreement with all previous posters.  The good news is you ran a factorial, so you might still have sufficient treatments for a fractional factorial. Here is my advice for missing data in an experiment.  

1. As datadad points out, what did you learn from the treatments which did not produce an output (this may be the most informative part of the experiment)?  Was it a result of the treatment or did something else happened?

2. Can you develop an alternative response variable?  Another way to measure the experimental units.

3. You might be able to use regression with the remaining data to create a model that will predict the missing data points (you will need to cognizant of DFs available)

4. Use the average of the other treatments for the missing data point (this tends to normalize that particular treatment effect)

5. Had you predicted the results á priori, use your predicted result.

6. Do 3-5 and substitute values for the missing data point.  perform fit model analysis and look to see how well those analysis agree.  If there is reasonable agreement, you can probably be confident in your analysis.  If not, you might need to re-run the missing treatments coupled with some you do have data for (to estimate block effect).