topic Re: Confidence Intervals using a Binomial Distribution in Discussions

Confidence Intervals using a Binomial Distribution

pmroz — Tue, 18 Jan 2011 20:47:25 GMT

I need to calculate 95% confidence intervals for the reporting fraction of adverse events at different time intervals. I'd like to use the "Exact Binomial Confidence Interval".

The link below shows how to do it using Excel; is there a comparable way to do this in JSL?

http://www.itl.nist.gov/div898/handbook/prc/section2/prc241.htm

Thanks!

Re: Confidence Intervals using a Binomial Distribution

Anonymous — Tue, 18 Jan 2011 22:23:44 GMT

It looks like that is using the Clopper-Pearson Exact CI. Here is what I have used for calculating the upper confidence interval for alpha. Unfortunately, the theory on how it was derived has faded from my memory.

Clopper Pearson Exact Upper Bound
(1 + (:Total - :Fail) / ((:Fail + 1) * F Quantile(
1 - :alpha,
2 * (:Fail + 1),
2 * (:Total - :Fail)
))) ^ (-1)

Re: Confidence Intervals using a Binomial Distribution

mpb — Tue, 18 Jan 2011 22:55:21 GMT

There are a couple of ways to imitate the Excel example directly using the JMP Binomial Distribution function. One way is to create a column with that formula and other columns for the independent variables and use the Profiler to achieve the target right hand side by varying the proportion p. The other way is to set it up as a nonlinear fit using p again to achieve the target right hand side. They both work.

Re: Confidence Intervals using a Binomial Distribution

ms — Wed, 19 Jan 2011 00:03:08 GMT

Here is an example of the nonlinear approach. The script below creates a data table with same data as in the Excel example. Run the table script to calculate the confidence limits.

(I first tried a neat solution that included the nonlinear platform (invisibly) in a column formula to enable automatic calculation. It worked, but not always as JMP sometimes got stuck inside an apparently infinite loop.)

Re: Confidence Intervals using a Binomial Distribution

pmroz — Wed, 19 Jan 2011 14:34:12 GMT

Thank you all for your answers. MS your solution worked when I pulled the New Script section out and ran it separately (in JMP 8.0.2 and 9.0.0).

Re: Confidence Intervals using a Binomial Distribution

pmroz — Wed, 19 Jan 2011 16:08:38 GMT

MS is there a way I can generalize your solution for a table full of
values? For example:

style="font-family: Courier New,Courier,monospace; font-weight: bold; text-decoration: underline;">Preferred
Term
Count     Total
Fraction

style="font-family: Courier New,Courier,monospace;">

style="font-family: Courier New,Courier,monospace;">Headache
18
513    3.5%

style="font-family: Courier New,Courier,monospace;">

style="font-family: Courier New,Courier,monospace;">Nausea
24
513    4.7%

style="font-family: Courier New,Courier,monospace;">

style="font-family: Courier New,Courier,monospace;">Dizziness          14
513    2.7%

style="font-family: Courier New,Courier,monospace;">

style="font-family: Courier New,Courier,monospace;">.

etc.

It appears from your solution that I would need to create a separate
one-row table for each row
to calculate the confidence limits.

Regards,

Peter

Re: Confidence Intervals using a Binomial Distribution

ms — Wed, 19 Jan 2011 16:34:25 GMT

Yes, I tried that, it worked but it was a bit shaky. I did not save the script but the principle was something like this:

I used script that excluded all rows then a for loop where each iteration unexcludes the current row, run the nonlinear script (incl. saving the parameter value), and finally re-excludes the current row. However, sometimes I got erroneous results probably because the row exclusion/unexclusion completion was running ahead of completion the numerical nonlinear fitting. One could try to find the sweet spot using Wait()-commands or use some other way to force completion.

Another possibility would be to run the non-linear with a by-argument using column with unique ID for each row. and then loop through the different nonlinear objects to finish and get estimates:
Something like

Re: Confidence Intervals using a Binomial Distribution

pmroz — Wed, 19 Jan 2011 20:31:00 GMT

I figured out a way to do it that might be faster than using the
nonlinear function. Takes about 12-18 tries in the loop
before finding an acceptable value. Adjust
the epsilon parameter to get higher or lower precision.
Takes less than a second on a table of 2400 values!

This was written in JMP 9 btw.

style="font-size: 10pt; font-family: "Courier New"; color: green;">/*

style="font-size: 10pt; font-family: "Courier New"; color: green;">Binomial
Confidence Intervals

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">Search
for upper and lower
confidence intervals

:
style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">.05
and .95

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">Uses
a table with columns called:

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">Case
Count
number of cases for this
Preferred Term

style="font-size: 10pt; font-family: "Courier New"; color: green;">Total
Cases
total number of cases

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">Add
two columns:

style="font-size: 10pt; font-family: "Courier New"; color: green;">

style="font-size: 10pt; font-family: "Courier New"; color: green;">Lower
CI Limit

style="font-size: 10pt; font-family: "Courier New"; color: green;">Upper
CI Limit

style="font-size: 10pt; font-family: "Courier New"; color: green;">*/
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">dt

style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">current
data table
style="font-size: 10pt; font-family: "Courier New"; color: black;">()
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">Format
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: purple;">"Percent"
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">10
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: black;">
)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style=""> formula(

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
max_iterations
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">100
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
epsilon
style="">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">.0001
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
lower_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">.0000000001
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
upper_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">.9999999999
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
target
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">.95
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
style="font-size: 10pt; font-family: "Courier New"; color: green;">//
Upper confidence interval
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">for
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(i

style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">
i
style="font-size: 10pt; font-family: "Courier New"; color: navy;"><=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
max_iterations
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">
i
style="font-size: 10pt; font-family: "Courier New"; color: navy;">++,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
test_b1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(lower_limit

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">
upper_limit)

style="font-size: 10pt; font-family: "Courier New"; color: navy;">/
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
bd
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">binomial
distribution
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">test_b1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Total
Cases
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Case
Count)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">if
style="font-size: 10pt; font-family: "Courier New"; color: black;">
((
style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">abs
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">bd

style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: black;">
target)

style="font-size: 10pt; font-family: "Courier New"; color: navy;"><=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
epsilon)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">break
style="font-size: 10pt; font-family: "Courier New"; color: black;">()
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: green;">//
then
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
upper_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
test_b1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: green;">//
else
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
lower_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
test_b1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
test_b1
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style=""> )
style="">

style="font-size: 10pt; font-family: "Courier New"; color: green;">//
formula
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
style="font-size: 10pt; font-family: "Courier New"; color: green;">//
end new column
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">dt

style="font-size: 10pt; font-family: "Courier New"; color: navy;"><<
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">new
column
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: purple;">"Upper
CI
Limit"
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">
numeric
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">
Continuous
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">Format
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: purple;">"Percent"
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">10
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: black;">
)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style=""> formula(

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
target
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">.05
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
style="font-size: 10pt; font-family: "Courier New"; color: green;">//
Upper confidence interval
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
test_b0
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(lower_limit

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">
upper_limit)

style="font-size: 10pt; font-family: "Courier New"; color: navy;">/
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
bd
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">binomial
distribution
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">test_b0
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Total
Cases
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Case
Count)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
upper_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
test_b0
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
lower_limit
style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
test_b0
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">
test_b0
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

Thanks again for everyone's suggestions and help!

Re: Confidence Intervals using a Binomial Distribution

mpb — Wed, 19 Jan 2011 21:20:38 GMT

The Chadd's formula and it's lower bound counterpart should give the same upper and lower bounds as the method in the nist manual. I'll look up the other bound if you are interested. That would require no coding at all except to create the column forumulas.

It may be worth mentioning that the "exact" calculation is not necessarily considered "best" in the sense that the resulting interval may be unnecessarily wide. Here are a couple of papers in PDF format that are readily available and which you may find interesting.

http://projecteuclid.org/DPubS/Repository/1.0/Disseminate?view=body&id=pdf_1&handle=euclid.ss/1009213286

http://www.stat.iastate.edu/preprint/articles/2002-05.pdf

Re: Confidence Intervals using a Binomial Distribution

Anonymous — Wed, 19 Jan 2011 21:30:24 GMT

+1 on that. I'm an Adjusted Wald fan, personally.

Re: Confidence Intervals using a Binomial Distribution

mpb — Wed, 19 Jan 2011 21:45:13 GMT

Ok, here are the upper and lower bound formulas for a two sided confidence interval:

style="font-size: 10pt; font-family: "Courier New"; color: black;">Pu

style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Total

style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail)

style="font-size: 10pt; font-family: "Courier New"; color: navy;">/
style="font-size: 10pt; font-family: "Courier New"; color: black;">
((
style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">*
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">F
Quantile
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">alpha

style="font-size: 10pt; font-family: "Courier New"; color: navy;">/
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">*
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">)))
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">^
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">Pl

style="font-size: 10pt; font-family: "Courier New"; color: navy;">=
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">
((
style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Total

style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">/
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail)

style="font-size: 10pt; font-family: "Courier New"; color: navy;">*
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: rgb(0, 0, 221);">F
Quantile
style="font-size: 10pt; font-family: "Courier New"; color: black;">(
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">
style="">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">2
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">*
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Total

style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">:
style="font-size: 10pt; font-family: "Courier New"; color: black;">Fail

style="font-size: 10pt; font-family: "Courier New"; color: navy;">+
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">,
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: black;">)))
style="font-size: 10pt; font-family: "Courier New"; color: black;">

style="font-size: 10pt; font-family: "Courier New"; color: navy;">^
style="font-size: 10pt; font-family: "Courier New"; color: black;">
(
style="font-size: 10pt; font-family: "Courier New"; color: navy;">-
style="font-size: 10pt; font-family: "Courier New"; color: teal;">1
style="font-size: 10pt; font-family: "Courier New"; color: black;">)
style="font-size: 10pt; font-family: "Courier New"; color: navy;">;

Note: When :Fail = 0 the lower limit is defined as 0. When :Fail = 1 the upper limit is defined as 1. Need to conditionally define the two column formulas.

Message was edited by: mpb

Re: Confidence Intervals using a Binomial Distribution

mpb — Wed, 19 Jan 2011 21:51:09 GMT

> +1 on that. I'm an Adjusted Wald fan, personally.

From the JMP docs:

The Confidence Intervals option generates the Score Confidence Intervals table, like the one shown here. The score confidence interval is recommended because it tends to have better coverage probabilities (1 - α), especially with smaller sample sizes (Agresti and Coull 1998).

Not sure if that's the same as the Adjusted Wald calcs. I should go back and read the references I listed! The first ref has some very interesting plots showing actual coverage.

Michael

Re: Confidence Intervals using a Binomial Distribution

Anonymous — Wed, 19 Jan 2011 21:58:21 GMT

Adj Wald is great for small sample sizes, and converges with most other methods with large ones. I have a couple papers *scanned* that discuss it, but a quick google search has some detail on it:
http://www.measuringusability.com/wald.htm