Analysis of DoE results with some defective parts

[#1]

Hi,

I am trying to model the results of the designed experiment where 30 parts were produced. The measured characteristics are various dimensions of the parts.

However, 10 among those parts have a clearly visible defect, so some dimensions for those defective parts are clearly wrong. It is possible that other dimensions are also affected.

There are 3 (4) possible ways that come to my mind how to do the modeling:

1) Remove the defective parts from the analysis (this is tricky since there are 10 of them).

1a) Remove only the dimensions that are clearly wrong, while assuming other dimensions are not affected.

2) Add another response to the model which describes the quality (1 for good, 0 for defective). Then target only those that have quality = 1

3) Add a blocking factor with random effect, and separate parts in 2 groups

What do you suggest? Do you think of any better way how to do this?

Thanks!

[#2]

Hi @ruskicar,

Based on your description, I suppose your DoE is model-based (platform Custom Design, DSD or Classical), not model-agnostic like Space-Filling designs ?

Removing points from a model-based design is complex and not recommended, as it could severely "damage" or bias the model obtained, since the points were created based on specific model's assumptions/hypothesis. You wouldn't have so much complexity/difficulty with model-agnostic design, as you have more flexibility about the modeling options.

There are some great ideas in your propositions, here are some comments or other propositions ("food for thought"). As I'm not a domain expert like you, please ignore any proposition that may sound irrelevant to your field :

1. Clearly not recommended, as you would completely remove information (even if biased) from your model and it could severely bias it.
   If there are values thresholds on the different dimensions you are measuring (above or below a certain threshold value, the data is probably erroneous), you may use the column property "Detection Limits" and specify the upper or lower threshold for the different responses ? Some use cases and examples here : Using Generalized Regression To Analyze Designed Experiments With Detection Limi... - JMP User Commu...
    
   Concerning 1.a), can you spot which dimensions seem wrong, from a domain expertise point of view, with possible support from different outliers techniques: univariate method like box-plots, multivariate analysis like T², Jacknife/Mahalanobis distances, PCA or clustering techniques like Normal Mixtures/K-Means ? If yes, you could introduce a new continuous numerical column (Freq/Weight) where you can put higher number for "valid" measurements and lower number for "erroneous" measurements, and use this column as "Freq" (frequency) property in the modeling platform used. This will put more emphasis on valid measurements than on non-valid measurements, as if valid measurements were measured several times more than non-valid measurements.
   You can have several way to impute these values depending on which analysis you used to spot erroneous measurement, but something interesting could work by using for example the reciprocal value of Jacknife/Mahalanobis distances as the calculated frequencies: the more distance for one experiment, the lower the frequency, so the less emphasis this experiment will have on the model. 
   
   
2. By using a binomial response (Safe/Anomalous), you could use a first model to see if there is a link between some factor values in your design and the presence of anomaly/strange values. If there is a link, you could save the prediction probabilities and use these probabilities as a second response in the optimization Profiler : optimize based on domain target (your initial response) with the maximum probability of being "safe" from anomalous values (2nd response based on probabilities from the binomial model).
   
   
3. It depends : The factor could be seen as either fixed and/or random effect, as having defective parts could have an impact on the mean response (fixed effect, even if not reproducible), but also as a random effect, as it could have an impact on response variance. If you can identify parts that are defective or not, you may be able to realize a short comparative test with the platform "Fit Y by X" in order to assess if the difference between parts is more oriented towards a difference in response mean, or towards a difference in response variance.

I hope this comment will help you and will give you some ideas

[#3]

Thanks Victor, those ideas will definitely help me.

[#4]

Sorry, I'm a bit confused.  There is not enough context to properly provide advice, but here are my thoughts:

If you are measuring dimensions, I assume these are continuous measurements.  Use these values to analyze the experiment.  One purpose of an experiment is to create variation in the response variables.  To some extent, the more variation you create, the easier it is to see causal relationships.  The purpose is not to create good parts.  In  fact it is to create variation in the parts over  a short period of time.  Since you have multiple dimensions, I would also do some multivariate analysis, but other than that, use fit model and load all of the dimensions into the Y, saturate the model to look for significant model effects. Reduce the model and analyze the residuals.

[#5]

Thanks statman, I agree with your argument, however:

- the parts are measured with the CMM.

- defective parts are deformed in such a way that for some dimensions the CMM probe can not reach the location (point) of measurement.