Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- JMP User Community
- :
- Discussions
- :
- Sample size for failure detection with 0 failures

News

On June 1, we’re asking you to select a content label when starting a new topic in the Discussions area. Read more to find out why.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

May 3, 2019 3:57 PM
(2626 views)

I have a population that I know the size of, let's say 50000. I have to manually check that values are correct. Now so far this has been 100% right after 14 samples, which took about 2 hours to do. Is there a way to either give the confidence of 0 failures at N samples, or how many samples I'd need to hit a specific alpha?

Vince Faller - Predictum

2 ACCEPTED SOLUTIONS

Accepted Solutions

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

You can use the hypergeometric distribution to derive sample sizes for these situations (finite population sizes), or the binomial for infinite population sizes. At 50,000 units, they will effectively give the same answer.

A rough, but actually pretty accurate heuristic way to do it is to use the rule of 3. This rule says to show the failure rate is no greater than (1/n) with approximately 95% confidence and no observed failures, you would need to have 3*n samples with 0 failures. For example, if I want to be 95% confident that the failure rate is no greater than 5% (1/20, therefore n=20), I would need to observe 60 samples with 0 failures.

If you go the more exact route, the answer is 59 samples.

-- Cameron Willden

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Here's a really simple function that figures it out iteratively:

```
zeroFailBinom = function({fail_rate,alpha},
p = 1; //initial value
i=1;
while( p > alpha,
p = binomial probability(fail_rate, i, 0);
i++;
);
print("Required Samples: "||char(i-1))
);
zeroFailBinom(0.01,0.05)
```

-- Cameron Willden

2 REPLIES 2

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

You can use the hypergeometric distribution to derive sample sizes for these situations (finite population sizes), or the binomial for infinite population sizes. At 50,000 units, they will effectively give the same answer.

A rough, but actually pretty accurate heuristic way to do it is to use the rule of 3. This rule says to show the failure rate is no greater than (1/n) with approximately 95% confidence and no observed failures, you would need to have 3*n samples with 0 failures. For example, if I want to be 95% confident that the failure rate is no greater than 5% (1/20, therefore n=20), I would need to observe 60 samples with 0 failures.

If you go the more exact route, the answer is 59 samples.

-- Cameron Willden

Highlighted

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Get Direct Link
- Email to a Friend
- Report Inappropriate Content

Here's a really simple function that figures it out iteratively:

```
zeroFailBinom = function({fail_rate,alpha},
p = 1; //initial value
i=1;
while( p > alpha,
p = binomial probability(fail_rate, i, 0);
i++;
);
print("Required Samples: "||char(i-1))
);
zeroFailBinom(0.01,0.05)
```

-- Cameron Willden