Re: Minimum Sample Size for a DOE to Reduce the Defect Rate?

sandeon · Oct 24, 2019 02:04 PM

I have a problem at my plant. In our assembly process, we are currently have a 15% defect rate, most likely caused by the process immediately upstream. I would like to run a DOE to discover the variables that affect the likelihood to failure. The batch size is around 150 pieces per work order. The ideal defect rate would be as low as reasonably possible but historically has been around 2 to 3%. I think the most relevant tool is the One Sample Proportion Calculator but I'm not exactly sure how to use it.

So, what I can't work out, is how many samples do I need to take per DOE experiment to be confident which variable affect the scrap rate? And how would I go about designing such a DOE based on defect rate?

Mark_Bailey · Oct 25, 2019 10:22 AM

If you are using JMP 14, then select Help > Books > Design of Experiments and see Chapter 17: Prospective Sample Size and Power. There is a section that explains this calculation and offers an example. This sample size is intended for a test of a hypothesized proportion defective. It asks a question about one population. Are you trying to demonstrate that the new failure rate is below 2%?

Also, are you counting the number of defective units or the number of defects in a sample of units? It makes a difference.

The design platforms, such as Custom Design, do not support a power analysis (another way of assessing sample size) for a binary response (pass/fail). Do you intend to design a multi-factor experiment or simply test some potential causes for the increase in the defect rate? If it is the latter case, then you could probably use the Two Sample Proportion Calculator to get a good idea of the sample size.

Some other experts more knowledgeable about these studies than me might have some other suggestions.

sandeon · Oct 25, 2019 12:28 PM

Thank you for your response.

@Mark_Bailey wrote:
If you are using JMP 14, then select Help > Books > Design of Experiments and see Chapter 17: Prospective Sample Size and Power. There is a section that explains this calculation and offers an example.

I just looked at the book, sometimes I feel a little thick when looking at examples like thesem so please bear with me. There is an example about two wafer assembly lines which I suppose could translate in my problem (a) current settings with a defect rate of 15% and (b) experiment settings with a defect rate of <15%.

If I try Two Proportion Sample Size calculator with a one-sided test (I think this is correct because I am only trying to detect if it is smaller, not larger). If I put proportion 1 = 0.15, proportion 2 = 0.02, the null difference = 0, alpha = 0.05 and power = 0.8, then I'm left with sample size 1 and 2 are equal to 56. Is this a correct application to this problem?

@Mark_Bailey wrote:
Are you trying to demonstrate that the new failure rate is below 2%?

Yes, but, not exactly. We won't know the exact failure rate of any of the experiments until we run them. We are trying to find the settings in the upstream process that reduce the failure/defect rate from its current level, that is <15% (but ideally, of course <2%, for example).

@Mark_Bailey wrote:
Also, are you counting the number of defective units or the number of defects in a sample of units? It makes a difference.

The number of defective units. There is only one defect this unit can have. The defective units are destroyed.

@Mark_Bailey wrote:
Do you intend to design a multi-factor experiment or simply test some potential causes for the increase in the defect rate? If it is the latter case, then you could probably use the Two Sample Proportion Calculator to get a good idea of the sample size.

This is what I wasn't sure about. If the sample size is so large, we need to kind of nudge the variables in the right direction. If it's small enough, then I would like to design a multi-factor experiment because I think there is some interaction between a few variables.

This is one of the more confusion problems I have worked on. It seems when the output signal is non-continuous, it really makes the problem more difficult!

Mark_Bailey · Oct 25, 2019 03:48 PM

I have a couple of questions that I should have asked before:

Is the determination of a defect based on a destructive test?
Is the definition of a defect based on a continous measure?

Yes, your result of n = 56 is correct assuming that the null difference is 0.

Well, if you are comparing two populations then this sample size calculation is correct.

The number of defective units is modelled with a binomial distribution. That is, you know both the number defective and the number non-defective.

Unfortunately, qualitative outcomes are much less informative than quantitiative responses. The dichotomous response is as bad as it gets.