BookmarkSubscribeRSS Feed
Choose Language Hide Translation Bar
vince_faller
Super User

Sample size for failure detection with 0 failures

I have a population that I know the size of, let's say 50000.  I have to manually check that values are correct.  Now so far this has been 100% right after 14 samples, which took about 2 hours to do.  Is there a way to either give the confidence of 0 failures at N samples, or how many samples I'd need to hit a specific alpha?

 

 

0 Kudos
2 ACCEPTED SOLUTIONS

Accepted Solutions
cwillden
Super User

Re: Sample size for failure detection with 0 failures

You can use the hypergeometric distribution to derive sample sizes for these situations (finite population sizes), or the binomial for infinite population sizes.  At 50,000 units, they will effectively give the same answer. 

 

A rough, but actually pretty accurate heuristic way to do it is to use the rule of 3.  This rule says to show the failure rate is no greater than (1/n) with approximately 95% confidence and no observed failures, you would need to have 3*n samples with 0 failures.  For example, if I want to be 95% confident that the failure rate is no greater than 5% (1/20, therefore n=20), I would need to observe 60 samples with 0 failures.

 

If you go the more exact route, the answer is 59 samples.

-- Cameron Willden
Highlighted
cwillden
Super User

Re: Sample size for failure detection with 0 failures

Here's a really simple function that figures it out iteratively:

zeroFailBinom = function({fail_rate,alpha},
	p = 1; //initial value
	i=1;
	while( p > alpha,
		p = binomial probability(fail_rate, i, 0);
		i++;
	);
	
	print("Required Samples: "||char(i-1))
);

zeroFailBinom(0.01,0.05)
-- Cameron Willden
2 REPLIES 2
cwillden
Super User

Re: Sample size for failure detection with 0 failures

You can use the hypergeometric distribution to derive sample sizes for these situations (finite population sizes), or the binomial for infinite population sizes.  At 50,000 units, they will effectively give the same answer. 

 

A rough, but actually pretty accurate heuristic way to do it is to use the rule of 3.  This rule says to show the failure rate is no greater than (1/n) with approximately 95% confidence and no observed failures, you would need to have 3*n samples with 0 failures.  For example, if I want to be 95% confident that the failure rate is no greater than 5% (1/20, therefore n=20), I would need to observe 60 samples with 0 failures.

 

If you go the more exact route, the answer is 59 samples.

-- Cameron Willden
Highlighted
cwillden
Super User

Re: Sample size for failure detection with 0 failures

Here's a really simple function that figures it out iteratively:

zeroFailBinom = function({fail_rate,alpha},
	p = 1; //initial value
	i=1;
	while( p > alpha,
		p = binomial probability(fail_rate, i, 0);
		i++;
	);
	
	print("Required Samples: "||char(i-1))
);

zeroFailBinom(0.01,0.05)
-- Cameron Willden