Sample Size and power for defect counts

[#1]

JMP version 17.2.0

We manufacture small units and mount 300 units on each assembly. Each assembly is tested, and we determine the number of defective units per assembly. We have records of 20 assemblies, and we know the number of defective units found in each of the 20 assemblies. A process change is considered, and we wish to determine whether the change reduces the defects per assembly significantly. What is the best practice to determine sample size, power, and reduction in defects: 1-sided test, target is 0 defects per assembly.

[#2]

Hi @MichaelR1:  I have a few questions/comments...in no particular order.

1. Is each unit made from same process (with the same defect rate), regardless of what assembly it is in?

2. Does assembly 1 get the first 300 units off the line, assembly 2 get the next 300 units, etc?

3. Is there an reason to think that the defect rate, on  average, is different from assembly to assembly (perhaps based how units are chosen for each assembly...see 2 above).

We can start with those 3... 

[#3]

Thanks for the questions. The units are produced in the same process. They are mounted on the assemblies in groups of 300: approximately FIFO. The assembly should not influence the defect rate, since the root cause was found in the unit production process.

As I understand now, this allows us to make the simple calculation: Power Explorer for One Sample Proportion

assumed proportion = current failure rate

alternative proportion = expected failure rate following process change

Is this the correct approach?

[#4 · Accepted solution]

Hi @MichaelR1 : I think so. Your null and alternative hypotheses are as follows.

H0: p=p0

Ha: p<p0

Where p0 is the current failure rate. Ha is what you are trying to prove (the new failure rate, p, is less than the current rate p0).

To "prove" Ha, p must truly be somewhere in Ha (<p0).  For example, if I assume p0=0.1, and expect p to be 0.05, and if I want my power to be 80%, and my type 1 error rate (alpha) to be 5%,  then in JMP it looks like the following.

So, if you take a random sample of 199 units, there is 80% chance of "proving" Ha (via rejecting H0 in favor of Ha). 

The type 1 error rate (Alpha) = 5% means that when H0 is true (p is not less than p0), there is a 5% chance of wrongly rejecting H0 in favor of Ha. Type 2 error = 1-Power= 0.2 (20%). Type 2 error then is the chance that you do not reject H0, even when Ha is true. You can choose Alpha and Power as you like, though 5% and 80% are common. And, I'm not a big fan of the Normal Approximation either; the Exact Test power plot looks weird due to the nature of pass/fail data. But normal approximations can be problematic especially when p is close to 0 or close to 1.

My two cents...