Hi,  my  name  is  Clay  Barker,  and  I'm  a statistical  developer  in  the  JMP  group.

I'm  going  to  be  talking  about  some  new features  in  the  distribution  platform

geared  towards  analyzing limit  of  detection  data.

It's  something  I  worked  on  with  my  colleague,  Laura  Lancaster,

who's  also  a  stat  developer  at  JMP.

To  kick  things  off, what  is  a  limited  detection  problem?

What  are  we  trying  to  solve?

It  is  basic  level,

a  limit  of  detection  problem  is  when   we  have  some  measurement  device

and  it's  not  able  to  provide  good measurements  outside  of  some  threshold.

That's  what  we  call a  limit  of  detection  problem.

F or  example,  let's  say  we  have, we're  taking  weight  measurements

and  we're  using  a  scale  that's  not  able to  make  measurements  below  1g .

In  this  instance,  we'd  say  that  we  have a lower  detection  limit  of  1

because we  can't  measure  below  that.

But  in  our  data  set,  we're  still recording  those  values  as  1.

Because  of  those  limitations,

we  see,  we  might  have  a  data  set that  looks  like  this.

In  part,  we  have  some  values  of  1  and some  non-1  values  that  are  much  bigger.

We  don't  really  believe that  those  values  are  1s.

We  just  know  that  those  are  at  most  1.

This  data  happens  all the  time  in  practice.

If  you  think  about  sometimes  we're  not able  to  measure  below  a  lower  threshold.

Sometimes  we're  not  able  to  measure above  an  upper  threshold.

Those  are  both  limited  detection  problems.

What  should  we  do  about  that?

Let's  look  at  a  really  simple  example.

I  simulated  some  data that  are  normally  distributed

with  mean  10  and  variance 1 ,

and  we're  imposing  a  lower detection  limit  of  9.

If  we  look  at  our  data  set  here,

we  have  some  values  above  9, and  we  have  some  9s.

When  we  look  at  the  histogram,

this  definitely  doesn't look  normally  distributed

because  we  have a  whole  bunch  of  extra  9s

and  we  don't have  anything  below  9.

What  happens  if  we  just model  that  data  as  is?

Well,  it  turns  out  the  results aren't  that  great.

We  get  a  decent  estimate of  our  location  parameter,  our  Mu,

it's  really  close to  10,  which  is  the  truth.

But  we've  really  underestimated that  scale  or  dispersion  parameter.

We've  estimated  it  to  be  0.8,

when  the  truth  is  that  we  generated it  with  scale  equal  to  1.

You'll  notice  that  our  confidence interval  for  that  scale  parameter

doesn't  even  cover  1.

It  doesn't  contain  the  truth  and  that's generally  not  a  great  situation  to  be  in.

What's  more,  if  we  look  at  the… We  fit  a  handful  of  distributions,

we  fit  the  log  normal, the  gamma,  and  the  normal,

well the  normal  distribution,  which  is what  we  use  to  generate  our  data,

it  isn't  even  competitive based  on  the  AIC.

Based  on  those  AIC  values,

we  would  definitely  choose  a  log  normal distribution  to  model  our  response.

We  haven't  done  a  good  job estimating  our  parameters.

We're  not  even  choosing to  use  the  distribution

that  we  generated  the  data  with.

W e  just  threw all  those  9s  into  our  data  set.

We  ignored  the  fact  that that was  incomplete  information

and that  didn't  work  out  well.

What  if,  instead  of  ignoring that  limit  of  detection,

what  if  we  just throw  out  all  those  times?

Well,  now  we've  got  a  smaller data  set  and  it's  biased.

We've  thrown  out  a  large chunk  of  our  data  on  it.

We  have  a  biased  sample  now.

Now  if  we  fit  our  normal  distribution,

now  we're  overestimating the  location  parameter,

and  we're  still  underest imating the  scale  parameter.

We're  actually  in  quite a  bad  position  still,

because  we  haven't  done  a  good job  with  either  of  those  parameters.

We're  still  unlikely  to  pick the normal  distribution.

Based  on  the  AIC, the  log  normal  and  the  gamma  distribution

both  fit  quite  a  bit  better than  the  normal  distribution.

We're  still  in  a  bad  place.

We  tried  throwing  out  the  9 s, and  that  didn't  turn  out  well.

We  tried  just  including  them  as  9 s.

That  didn't  turn  out  well  either.

What  should  we  do  instead?

The  answer  is  that  we  should  treat

those  observations at  the  limit  of  detection.

We  should  treat  those  as censored  observations.

Censoring  is  a  situation  where

we  only  have  partial  information about  our  response  variable.

That's  exactly the  situation  we're  in  here.

If  we  have  an  observation  at  the  lower detection  limit,  and  here  I've  denoted  it

D sub L,

we  say  that  observation is  less  censored.

We  don't  say  that  Y  is  equal to  that  limit  of  detection.

We  say  that  Y  is  less  than or  equal  to  that  DL  value.

On  the  flip  side,

if  we  have  a  upper  limit  of  detection, denoted  DU  here,

those  observations  are  right  censored.

Because  we're  not saying  that  Y  is  equal  to  that  value.

We're  just  saying  it's at  least  that  value.

If  you're  looking  for  more  information about  how  to  handle  censored  data,

one  of  the  references that  we  suggest  all  the  time  is

Meeker and  Escobar's  book Statistical  Methods  for  Reliability  Data.

That's  a  really  good  overview for  how  you  should  treat  censored  data.

If   you've  used  some  of  the...

If  you  use  some  of  the  features  and the  survival  and  reliability  menu  in JMP,

then  you're  familiar  with  things like  life  distribution  and  fit  life  by  X.

These  are  all  platforms that  accommodate  censoring  in  JMP.

What  we're  excited  about  in  JMP  17  is  now

we  have  added  some  features to  distribution  so  that  we  can  handle

this  limit  of  detection  problem and distribution  as  well.

All  you  have  to  do  is  you  add

a  detection  limit  column  property to  your  response  variable,

and  you  specify  what  the  upper and  or  lower  detection  limit  is,

and  you're  good  to  go, there's  nothing  else  you  have  to  do.

In  my  simulated  example, I  had  a  lower  detection  limit  of  9.

I  would  put  9  in  the  lower detection  limit  field  here.

That's  really  all  you  have  to  do.

By  specifying  that  detection  limit,

now  distribution  is  going  to  say,   okay,  I  know  that  values  of  9

are  actually  left  censored,

and  I'm  going  to  do estimation  accounting  for  that.

Now  with  that  same  simulated  example, and  this  lower  detection  limit  specified,

now  you'll  notice  we  get  a  much  more reasonable  fit  the  normal  distribution.

Now  our  confidence  interval  for  both the  location  and  scale  parameter

covers  the  truth,

because  we  know,  again,  the location  was  10  and  the  scale  was  1.

Now  our  confidence intervals  cover  the  truth

and that's  a  much  better  situation.

If  you  look  at  the  CDF  plot  here,

this  is  a  really  good  way  to  compare our  fitted  distribution  to  our  data.

What  it's  doing  is  that  red  line is  the  empirical  CDF,

and  the  green  line  is  the  fitted  normal  CDF.

as  you  can  tell,  they're really  close  up  until  9.

And that  makes  sense,  because that's  where  we  have  censoring.

We're  doing a  much  better  job  fitting  these  data

because  we're  properly handling  that  detection  limit.

I  just  wanted  to  point  out  that  when you've  specified  the  detection  limit,

the  report  makes  it  really clear  that  we've  used  it.

As  you  can  see  here,

it  says  the  fitted  normal  distribution with  detection  limits,

and  it  lets  you  know  exactly which  detection  limits  it  used.

Now  not  only  are,

because  we're  doing  a  better  job estimating  our  parameters,

things  like  inference  about  those parameters  is  more  trustworthy.

If  we  do  something  like we  look  at  the  distribution  profiler

now  we  can  trust  these

inference  based on  our  fitted  distribution,

we  feel  much  better  about  trusting  things like  the  distribution  profiler.

With  the  simulated  example, if  we  use  our  fitted  normal  distribution,

Because  we  properly   handled censoring,

we  know  that  about  16 % of  the  observations

are  falling below  that  lower  detection  limit.

I  also  wanted  to  point  out  that

when  you  have detection  limits  in  distribution,

now  we're  only  able  to  fit a  subset  of  the  distributions

that  you  would  normally see  in  the  distribution  platform.

We  can  fit  the  normal,  exponential, gamma  log,  normal,  WI  and  beta.

All  of  those  distributions support  censoring

or  limited detection  in  distribution.

But  if  you  were  using  something  like the  mixture  of  normals,

well,  that   that  doesn't extend  well  to  sensor  data.

You're  not  going  to  be  able  to  fit  that   when  you  have  a  limit  of  detection.

I  also  wanted  to  point  out if  you  have  JMP  pro

and  you're  used  to  using the  generalized  regression  platform,

generalized  regression  recognizes that  exact  same  column  property.

The  detection  limit  column  property

is  recognized  by  both distribution  and  generalized .

One  of  the  really  nice  things  about

this  new  feature  is  that  it  gets  carried on  to  the  capability  platform.

If  you  do  your  fit  and  distribution, and  you  launch  capability,

now  we're  going  to  get  more trustworthy  capability  results.

Let's  say  that  we're manufacturing  a  new  drug,

and  we  want  to  measure the  amount  of  sum  impurity  in  the  drug.

Our  data  might  look like  what  we  have  here.

We  have  a  bunch  of  small  values,  and  we have  a  lower  detection  limit  of  1  mg.

these  values  of  1 that  are  highlighted  here,

we  don't really  think  those  are  1.

We  actually  think  it's  something... We  know  that  it's  something  less  than 1 .

We  have  an  upper  specification limit  of  2.5  milligrams.

this  is  a  situation  where  we  have both  spec  limits  and  detection  limits.

It's  really  easy  to  specify those  in  the  column  properties.

Here  we've  defined  our  upper  spec  limit as  2.5

And  our  lower detection  limit  of  1.

Now  all  you  have  to  do  is  just

give  distribution the  column  that  you  want  to  analyze.

It  knows  exactly  how  to  handle  that response  variable.

Let's  look  at  the  capability  results.

Now,  because  we've  properly handled  that  limit  of  detection,

we  trust  that  our  log  normal  fit  is  good.

We  see  that  our  Ppk, value  here  is  0.546 .

That's  not  very  good.

Usually  you  would  want  a  Ppk  above  1.

This  is  telling  us  that our  system  is  not  very  capable.

We've  got  some  problems  that  we might  need  to  sort  out.

Once  again,  what  would  have  happened  if we  had  ignored  that  limit  of  detection

and  we  had  just  treated  all  those 1s  as  if  they  truly  were  1s.

Well,  let's  take  a  look.

We  do  our  fit,  ignoring  the  limit  of detection,  and  we  get  a  Ppk  of  above  1.

Based on  this  fit,

we  would  say  that  we  actually  have a  decently  capable  system,

because  a  Ppk  of  1  is  not  too  bad.

It  might  be  an  acceptable  value.

By  ignoring  that  limit  of  detection,

we've  tricked  ourselves  into  thinking  our system  is  more  capable  than  it  really  is.

I  think  this  is  a  cool  example,

because  we  have  a  lower  detection  limit, which  may  lead  you  to  believe,

well,  I  might  be  maybe  ignoring  the  limit of  detection  would  be  conservative,

because  I'm  overestimating the  location  parameter.

That's  true, when  we  ignore  the  limit  of  detection,

we're  overestimating that  location  parameter.

But  the  problem  is  we're  grossly underestimating  the  scale  parameter.

That's  what  makes  us  make  bad decisions  out  in  the  tail

of  that  distribution.

By  ignoring  that  limit  of  detection,

we've  really  gotten  ourselves into  a  bad  situation.

Just  to  summarize,

it's  really  important  to  recognize  when our  data  feature  a  limit  of  detection.

I  think  it's  easy  to  think  of,

sometimes  we  think  about  data  sets  where

maybe  we've  analyzed the  response  as  is  in  the  past,

when  really,  maybe  we  should  have adjusted  for  a  limit  of  detection.

Because  like  we  just  saw,  when  we  ignore those  limits,  we  get  misleading  fits.

Then  we  may  make  bad  decisions based  on  those  misleading  fits.

Like  we  saw  in  our  example,

we  got  bad  estimates or  the  location  and  scale  parameters,

and  our  Ppk  estimate  was almost  double  what  it  should  have  been.

But  what  we're  excited  about  in  JMP  17

is  that  the  distribution  platform  makes  it really  easy  to  avoid  these  pitfalls

and  to  analyze  this  kind of data  properly.

All  you  have  to  do  is  specify that  detection  limit  column  property,

and  distribution  knows exactly  what  to  do  with  that.

Today  we  only  looked  at lower  detection  limits,

but  you  can  just  as  well have  upper  detection  limits  as  well.

In  fact,  you  can  have  both.

Like  I  said,  there's  only six  distributions  that  currently  support

censoring inside  of  the  distribution  platform.

But  those  are  also  the  six  most  important distributions  for  these  kinds  of  data.

It  really  is a  good selection  of  distributions.

That's  it.

I just wanted  to  thank  you  for  your  time

and  encourage  you  to  check  out  these enhancements  to  the  distribution  platform.

Thanks.