Choose Language Hide Translation Bar

Improved Add-In for confidence limits for combinations of variance components in Mixed Models (2020-US-30MP-623)

Hadley Myers, JMP Systems Engineer, SAS
Chris Gotwalt, JMP Director of Statistical Research and Development, SAS

 

The need to determine confidence intervals for linear combinations of random mixed-model variance components, especially critical in Pharmaceutical and Life Science applications, was addressed with the creation of a JMP Add-In, demonstrated at the JMP Discovery Summit Europe 2020 and available at the JMP User Community. The add-in used parametric bootstrapping of the sample variance components to generate a table of simulated values and calculated “bias-corrected” (BC) percentile intervals on those values. BC percentile intervals are better in accounting for asymmetry in simulated distributions than standard percentile intervals, and a simulation study using a sample data set at the time showed closer-to-true α-values with the former. This work reports on the release of Version 2 of the Add-In, which calculates both sets of confidence intervals (standard and BC percentiles), as well as a third set, the “bias-corrected and accelerated” confidence interval, which has the advantage of adjusting for underlying higher-order effects. Users will therefore have the flexibility to decide for themselves the appropriate method for their data. The new version of the Add-In will be demonstrated, and an overview of the advantages/disadvantages of each method will be addressed.

 

 

Auto-generated transcript...

 

Speaker

Transcript

Hello, my name is Chris Gotwalt
00 08.966
3
has been developed for variance
components models, we we think
00 25.566
7
statistical process control
program, one has to understand
00 40.466
11
ascertain how much measurement
error is attributable to testing
00 55.500
15
there might be five or 10 units
or parts tested per operator,
00 10.766
19
different measuring tools is
small enough that differences in
00 26.033
23
measurement to measurement,
repeatability variation, or
00 39.900
27
measurement systems analyses, as
well as a confidence interval on
00 52.766
31
interval estimates in the report
and obtain a valid 95% interval
00 07.033
35
calculate confidence intervals,
because we believed it would be
00 23.400
39
and the sum of the variance
components. Unfortunately, the
00 38.266
43
r&r study. So because variance
components explicitly violate
00 57.300
47
you were to use the one click
bootstrap on variance components
00 10.566
51
less. So when we were designing
fit mixed, and the REML
00 27.933
55
independent. So back to the
drawing board. So it turns out
00 44.666
59
in JMP. One approach is called
the parametric bootstrap that
00 01.333
63
comparison of the two kind of
families of bootstrap. So the
00 18.333
67
they're, they're not assuming
any underlying model. And it's
00 37.200
71
the rows in the data table are
independent from one another.
00 52.766
75
values, it has the advantage
that we don't have to make this
00 09.866
79
bootstrap simulation. The
downside to this is that you
00 25.133
83
do a quick introduction to what
the bootstrap...the parametric
00 41.966
87
to identify or wanted to
estimate the crossing time of a
00 04.733
91
162.8. Now, we want to use a
parametric bootstrap to to go
00 22.466
95
has the ability to save the
simulation formula back to the
00 35.933
99
that uses the estimates in the
report as inputs into a random
00 53.300
00.666
104
And we take our estimates and
pull them out into a separate
00 17.666
108
And then what we have can be
seen as a random sample from the
00 37.000
112
formula column for the crossing
time. And that is automatically
00 53.900
116
those...on the crossing time, or
any quantity of interest. When
00 15.366
120
simulation, create a formula
column of whatever function of
00 28.366
124
derive quantity of interest and
obtain confidence intervals
00 47.233
128
the add in so that you're able
to do this quite easily for
00 05.033
132
133
we'll start by showing you how
to run the add in yourself once
00 25.500
137
first version was presented at
the JMP 2020 Discovery Summit
00 42.566
141
overview, but we'll show you the
references where you can dive in
00 58.866
145
perfectly fine as well. So I'm
going to go ahead and start with
00 14.700
149
makes use of the fit mixed
platform, right, created from
00 31.333
153
the add in will only work with
JMP Pro. So someone might,
00 49.066
157
want some measure like
reproducibility. So that would
00 10.166
161
as we said, to calculate the
estimate for these, there's no
00 26.066
165
columns here. The reality is
much, much, much more
00 43.066
169
of the estimate without
considering the worst case
00 59.233
173
production that the actual
variance is higher than they have
00 19.733
177
don't risk being out of spec in
production. So to run the add in
00 35.700
181
From here, I can select the
linear combination of confidence
00 55.266
185
simulations, you get a better
estimate of the confidence
00 10.500
189
2500. I'm going to leave it as
1000 here just for demonstration
00 28.733
193
operator or the batch variable,
and then press perform analysis.
00 45.533
197
purpose of this demonstration, I
think I will stop it early.
00 07.733
201
calculated confidence limits, the
bootstrap quantiles, which are
00 28.933
205
these two tabs. But if you'd
like to see how those compare,
00 42.366
209
so what does enough mean, enough
for your confidence limits to
00 57.400
213
stopped it before a thousand. So
that's how the add in works. And
00 15.466
217
distributed around the original
estimate, they are in fact
00 37.366
221
relaunch this analysis. So
you'll see that when the
00 56.433
225
European Discovery, required
bounded variance confidence
00 16.766
229
that, if that happens for some
of the bootstrap samples or for
00 40.466
233
early, again, I'll just let it
run a little bit. Yeah, so I, as
00 00.966
237
the samples are allowed, in some
cases, to be below zero. So in
00 28.400
242
simulation column here, this
column of simulated
00 49.100
246
see them both at the same time.
It's a bit... it's a bit tricky,
00 16.300
252
right components, it's a good
idea to run the add in directly
00 31.766
256
that column is then deleted. So
one thing to to mention, before
00 50.500
260
accounts for the skewness of the
bootstrap distributions, right,
00 14.200
264
that. And then the accelerated
takes that even further. So here
00 27.200
268
thing to mention is that the
alpha in this represents the
00 43.700
272
value for which it's been
calculated? And what can we do to
00 03.233
276
up to investigate the four
different kinds of the variance
00 24.700
280
method, the bias corrected
method and the BCa. We also
00 43.000
284
study. So for all 16
combinations of these three
00 01.566
288
combinations of confidence
intervals, and kept track of how
00 20.400
292
293
coverage as we're varying these
three variables, and we see here
00 45.300
297
298
techniques. And the second best
is the bias corrected and
00 09.966
302
the best one. Now, if you turn
no bounds on, which means that
00 28.433
306
variance components with a
pretty close to 95% coverage.
00 48.200
310
intervals are performing
similarly at about 93%. But
00 02.200
07.800
315
to what a master's thesis paper's
research would have, would
00 27.966
319
potentially more work to be
done. There's other interval
00 42.566
323
things like generalized
confidence intervals. General
00 59.466
327
intervals might also do the
trick for us as well. Hadley's
00 19.966
331
so that you can now do
parametric bootstrap simulations
00 37.566
335
16. When you bring that up, you
can enter the linear combination
00 51.766
339