Welcome  to  our  presentation  on  BPC- Stat:

Accelerating  process  characterization

with  agile  development of  an  automation  tool  set.

We'll  show  you   our  collaborative  journey

of  the  Merck  and  Adsurgo development  teams.

This  was  made  possible  by  expert  insights, management sponsorship,

and  the  many  contributions from  our  process  colleagues.

We  can't  overstate  that  enough.

I'm  Seth  Clark.

And I'm  Melissa  Matzke.

We're  in  the  Research  CMC  Statistics  group at  Merck  Research  Laboratories.

And  today,  we'll  take  you  through   the  problem we were challenged with,

the  choice  we  had to make,

and  the  consequences of  our  choices.

And  hopefully,  that  won't  take  too  long and we'll quickly get to the best part,

a  demonstration   of  the  solution  BPC- Stat.

So  let's  go  ahead  and  get  started.

The  monoclonal  antibody  or  mAb   fed- batch process

consists  of  approximately 20 steps or  20  unit  operations

across  the  upstream  cell  growth  process and  downstream  purification  process.

Each  step  can  have up  to  50  assays

for  the  evaluation  of  process and  product  quality.

Process  characterization  is  the  evaluation of  process  parameter  effects  on  quality.

Our  focus  is  the  statistics  workflow

associated  with  the  mAb process  characterization.

The  statistics  workflow  includes study design  using  Sound  DOE  principles,

robust  data  management, statistical  modeling  and  simulation.

This  is  all  to  support   parameter classification

and  the  development of  the  control  strategy.

The  goal  for  the  control  strategy   is to make statements

about  how  the  quality is  to  be  controlled,

to  maintain  safety  and  efficacy  through consistent  performance  and  capability.

To  do  this,  we  use  the  statistical models developed  from  the  design  studies.

Parameter  classification  is  not a statistical analysis,

but  it  can  be  thought  of   as  an  exercise

of translating statistically meaningful  to  practically  meaningful.

The  practically  meaningful  effects

will be used to guide and inform SME  (subject matter  expert)  decisions

to  be  made  during  the  development  of  the  control  strategy.

And  the  translation  from  statistically to practically meaningful

is  done  through   a  simple  calculation.

It's  the  change  of  the  attribute  mean, that is the parameter effect,

relative  to  the  difference  in  the  process mean

and  the  attribute  acceptance  criteria.

And  depending  on  how  much  of the acceptance criteria range

gets used by  the  parameter  effect

determines whether that  process  has a practically  meaningful  effect  on  quality.

So  we  have  a  defined  problem -

the  monoclonal antibody  PC statistics workflow

and  study  design   through  control  strategy.

How  are  we  going  to  implement the  statistics  workflow  in  a  way

that  the  process  scientists  and  engineers will  actively  participate  in

and own these components  of  PC, allowing us, the  statisticians,

to provide oversight and  guidance   and  allow  us  to  extend  our  resources.

We  had  to  choose  a  statistical  software that included data management,

DOE, plotting,   linear  mix  model  capabilities.

Of  course,  it was extendable  through  automation,

intuitive, interactive  and  fit- for- purpose without  a  lot  of  customization.

And  JMP  was  an  obvious clear  choice  for  that.

Why?

Because  it  has  extensive customization and  automation  through  JSL;

many  of  our  engineers,

it's already their current  go- to  software for  statistical  analysis;

we  have  existing  experience   training in our organization;

and  it's  an  industry- leading  DOE   with  data  interactivity.

The  profiler  and  simulator  in  particular is  very  well  suited  for  PC  studies.

Also,  the  scripts  that  are  produced  are standard,  reproducible,  portable  analysis.

We're  using  JMP   to  execute  study design,

data  management, statistical  modeling  and  simulation

for  one  unit  operation  or  one  step that  could  have  up  to  50  responses.

The  results  of  the  analysis   are  moved to a study report

and  used  for  parameter  classification.

And  we  have  to  do  all  these  20  times.

And   we have  to  do  this   for  multiple  projects.

So  you  can  imagine it's a  huge  amount  of  work.

When  we  started  doing  this initially,  we  were  finding  that  we  were doing

a  lot  of  editing  of  scripts before  the  automation.

We  had  to  edit  scripts  to  extend  a copy existing analysis to other responses,

we  had  to  edit  scripts   to  add  where conditions,

we  had  to  edit  scripts to  extend  the  models  to  large  scale.

We  want  to  stop  editing  scripts.

Many  of  you  may  be  familiar  with  the simulator  set up  can  be  very  tedious.

You  have  to  set  the  distribution for  each  individual  process  parameter.

You  have  to  set  the  add  random  noise,

and  you  have  to  set  the  number of  simulation  runs  and  so  on  and  so  on.

There  are  many  different  steps,

including  the  possibility   of  editing scripts if we want to change

from  an  internal  transform  to  the  explicit  transform.

The  simulator  is  doing   add  random  noise

on  a  log- normal  type  basis, for  example,   the  log  transform.

We  want  to  stop  that  manual  set up.

Our  process  colleagues  and  us we're spending enormous time

compiling information   for  the  impact  ratio calculations

that  we  use  to  make  parameter   classification  decisions.

We  are  using  profiles  to  verify  those

and  assembling  all  this  information into  a  heat  map

that  then  would  be   very  tedious  exercise

to  decompose  the  information back  to  the  original  files.

We  want  to  stop  this  manual parameter  classification  exercise.

Of  course,  last  but  not  least, we  have  to  report our  results.

And  the  reporting  involves  copying,

or  at  least  in  the  past  involved copying and  pasting  from  other  projects.

And  then,  of  course,   you  have  copy  and paste errors,

copy  and  pasting  from  JMP,

you  might  put  the  wrong  profile or  the  wrong  attribute  and  so  on.

We  want  to  stop   all  this  copying  and  pasting.

We  clearly  had  to  deal  with  the consequences  of  our  choice  to  use  JMP.

The  analysis  process  was  labor- intensive, taking weeks, sometimes months

to finish an  analysis   for  one  unit  operation.

It  was  prone  to  human  error and  usually  required  extensive  rework.

It  posed  to  be  an  exceptional challenge to  train  colleagues  to  own  the  analysis.

We  developed  a  vision   with  acceleration in mind

to  enable  our  colleagues

with a  standardized   yet  flexible  platform approach

to  the  process  characterization and  statistics  workflow.

So  we  had  along  the  way some  guiding  principles.

As  we  mentioned  before,   we  wanted  to stop editing JMP scripts

so  any  routine  analysis,   no  editing  is  required.

And  our  JMP  analysis, they need to stand on their own,

they  need  to  be  portable   without  having to install BPC-Stats

so  that  they  live  on  only  requiring   the  JMP  version  they  were  used.

We  collected  constant  feedback, we  constantly  updated,

we  constantly  tracked  issues relentlessly,  updating  sometimes  daily,

meeting  weekly  with   Adsurgo our development  team.

Our  interfaces,  we  made  sure that  they  were  understandable.

We  use  stoplight  coloring   such  as  green for good,

yellow  for  caution, and  issues  flagged  with  red.

We  had  two  external  approaches   or  inputs  into  the  system,

an  external  standardization, which  I'll  show  a  bit  later,

where  the  process  teams

define  the  standard  requirements  for  the  analysis file.

And  our  help  files, we  decided  to  move  them  all  externally

so that  they  can  continue  to  evolve as the  users  in  the  system  evolved.

We  broke  our  development   into  two  major development cycles.

Early  development  cycle,

where  we  develop  proof  of  concepts so we would  have  a  problem,

we  would  develop  a  proof  of  concept working  prototype  to  address  that  problem.

We  would  iterate  on  it   until  it  was solving the problem

and  it  was  reasonably  stable.

And  then  we  moved  it   into  the  late development cycle

and  continued  the  agile  development, where  in  this  case,

we  applied  it  to  actual  project  work and  did  very  careful  second- person  reviews

to  make  sure  all  the  results  were  correct and  continue  to  refine  the  modules

based on  feedback  over  and  over  again to get to  a  more  stable  and  better  state.

One  of  these  proof  of  concept  wow factors that  we  did  at  the  very  beginning

was  the  Heat  Map  tool,

which brought together all kinds of  information  in  the  dashboard

and it saved  the  team's enormous amounts  of  time.

I'll  show  you  an  example  of  this later.

But  you  can  see  the  quotes on  the  right- hand  side,

they  were  very  excited  about  this. They  actually  helped  design  it.

And  so  we  got  early  engagement, we  got  motivation  for  additional  funding

and  a  lot  of  excitement  generated by  this  initial  proof  of  concept.

In  summary,  we  had  a  problem  to  solve, the PC statistics workflow.

We  had  a  choice  to  make   and  we  chose  JMP.

And  our  consequences  copying  and  paste, manual,  mistakes,  extensive  reworking.

We  had  to  develop  a  solution, and  that  was   BPC-Stat.

I'm  extremely  happy   to  end  this  portion of our talk

and  let  Seth  take  you  through   a  demonstration  of   BPC-Stat.

We  have  a  core  demo   that  we've  worked  out

and  we're  actually  going to  begin  at  the  end.

In  the  end,  which  is  that   proof of  concept  Heat  Map tool

that  we  had  briefly  shown  a  picture of is  when  the  scientists  have  completed

all  of  their  analysis and  the  files  are  all  complete,

the  information  for  impact  ratio  is  all available

and  collected into  a  nice  summary  file,

which  I'm  showing  here.

Where  for  each  attribute in  each  process  factor,

we  have  information  about  the  profiler,

it's  predicted  means  and  ranges as  the  process  parameter  changes.

So  we  can  compute  changes  across that  process  factor  in  the  attribute

and  we  can  compute  that  impact ratio that  we  mentioned  earlier.

Now,  I'm  going  to  run  this  tool and  we'll  see  what  it  does.

So  first,  it  pulls  up  one  of  our early  innovations  by  the  scientists.

We  organized  all  this  information that  we  had  across  multiple  studies.

Now,  this  is  showing three different process  steps  here

and you can see on  the  x- axis, we have the  process  steps  laid  out.

In  each  process  step,  we  have  different studies  that  are  contributing  to  that,

we  have  multiple  process  parameters,

and  they  are  all  crossed   with these attributes  that  we  use

to assess the  quality  of  the  product that's  being  produced.

And  we  get  this  Heat  Map  here.

The  white  spaces  indicate  places

where the  process  parameter  either  dropped out  of  the  model  or  was  not  assessed.

And  then  the  colored  ones  are  where we  actually  have  models  built.

And  of  course,   the  intensity  of  the  heat

is  depending  on  this practical  impact  ratio.

This  was  great  solution for  the  scientists,

but  it  still  wasn't  enough  because we had  disruptions  to  the  discussion.

We  could  look  at  this  and  say,  okay, there's  something  going  on  here,

we  have  this  high  impact  ratio,

then  they  would  have  to  track down  where  is  that  coming  from.

Oh,  it's  in  this  study. It's  this  process  parameter.

I  have  to  go  to  this  file.

I  look  up  that  file, I  find  the  script  among  the  many  scripts.

I  run  the  script,

I  have  to  find  the  response and then finally  I  get  to  the  model.

We  enhance  this  so  that  now  it's  just a simple  click  and  the  model  is  pulled  up.

The  relevant  model  is  pulled  up  below.

You can  see  where  that  impact  ratio of  one  is  coming  from.

Here, the  gap  between   the  predicted  process  mean

and  the  attribute  limit is  the  space  right  here.

And  the  process  parameter  trend is taking  up  essentially  the  entire  space.

That's  why  we  get that  impact  ratio  of  one.

Then  the  scientists  can  also   run their simulators

that  have  been  built  or  are  already  here,  ready  to  go.

They  can  run  simulation  scenarios to  see  what  the  defect  rates  are.

They  can  play  around   with  the  process parameters

to  establish  proven  acceptable ranges that  have  lower  defect  rates.

They  can  look  at  interactions, the  contributions  from  that.

They  can  also  put  in  notes  over  here on the right

and  save  that  in  a  journal file

to  keep  track  of  the  decisions  that they're making.

And  notice  that  all  of  this   is  designed  to  support  them,

maintaining  their  scientific  dialogue, and prevent  the  interruption  to  that.

They  can  focus  their  efforts on  particular  steps.

So  if  I  click  on  a  step, it  narrows  down  to  that.

Also,  because  our  limits   tend  to  be in flux,

we  have  the  opportunity  to  update those

and  we  can  update  them  on  the  fly  to  see  what  the  result  is.

And  you  can  see  here  how  this  impact changed

and  now  we  have  this  low- impact ratio

and  say,  how  does  that  actually   look  on  the  model?

The  limit's  been  updated now, you  can  see  there's  much  more  room

and  that's  why  we  get  that  lower  impact ratio  and  we'll  get  lower  failure  rates.

That  was  the  heat  map  tool   and  it  was a huge win

and  highly  motivated  additional investment  into  this  automation.

I  started  at  the  end,

now  I'm  going  to  move  to  the  beginning  of the process statistics workflow,

which  is  in  design.

When  we  work  with  the  upstream  team,

they  have  a  lot  of  bio reactors that  they  run  in  each  of  these  runs.

This  is  essentially a  central  composite  design.

Each  of  these  runs  is  a  bio reactor and  that  bio reactor  sometimes  goes  down

because  of  contamination  or  other  issues that  essentially  are  missing  at  random.

So  we  built  a  simulation

to evaluate this to  potential  losses called  evaluate  loss  runs.

And  we  can  specify   how  many  runs  are  lost.

I'm  just  going  to  put  something   small  here for demonstration

and  show  this  little  graphic.

What  it's  doing  is  it's  going   through randomly selecting points

to  remove  from  the  design   in calculating  variance inflation factors,

which  can  be  used   to  assess  multi linearity

and  how  well  we  can  estimate   model  parameters.

And  when  it's  done, it  generates  a  summary  report.

This  one's  not  very  useful   because  I  had very few simulations,

but  I  have  another  example  here.

This  is  500  simulations on  a  bigger  design.

And  you  can  see we get this  nice  summary  here.

If  we  lose  one  bio reactor, we have essentially a zero probability of

getting  extreme  variance  inflation factors or non- estimable  parameter  estimates.

And  so  that's  not  an  issue.

If  we  lose  two  bio reactors  up  to  about 4%,  that's  starting  to  become  an  issue.

So  we  might  say  for  this  design, two  bio reactors  is  a capability  of  loss.

And  if  we  really  wanted   to  get  into  the  details,

we  can  see  how  each  model  parameters impacted  on  variance  inflation

for given number  of  bioreactors  lost,

or  we  can  rank  order  all  the  combinations of bioreactors that are lost,

which  specific  design  points   are  impacting the design the most,

and  do  further  assessments  like  that.

That's  our  simulation  and  that's  now a routine  part  of  our  process  that  we  use.

I'm  going  to  move  on  here   to  standardization.

We  talked  about  the  beginning  of the process statistics workflow,

the  end  of  the  process   statistics  workflow,

now  I'm  going  to  go  back  to  what is the  beginning  of  BPC-Sta t  itself.

When  people  install  this, they  have  to  do  a  setup  procedure.

And  the  setup  is  basically specifying  the  preferences.

It's  specifying  those  input parameter,

input  standardizations that we had  talked  about  earlier,

as  well  as  the  help  file, what  process  area  they're  working  with,

and  default  directory that  they're  working  with.

And  then  that  information  is  saved  into the  system  and  they  can  move  forward.

Let  me  just  show  some  examples  here  of  the standardization  files  and  the  Help  file.

Help  file  can  also  be  pulled up under  the  Help  menu.

And  of  course,  the  Help  file is  the  JMP  file  itself.

But  notice  that  it  has   in  this  location column,

these  are  all  videos  that  we've created that  explain  and  they're  all timestamped

and  so  users  can  just  figure  out   what  they're  looking  for,  what  feature.

Click  on  it,  immediately pull  up  the  video.

But  what's  even  more  exciting  about  this  is in all the dialogues of BTC-Stat,

when  we  pull  up  a  dialogue   and  there's a Help button there,

it  knows  which  row  in  this  table  to  go  to  get  the  help,

and  it  will  automatically  pull  up.

If  I  click  that  Help  button, it will automatically pull up

the  associated  link  and  training   to  give  immediate  help.

That's  our  Help  file.

The  standardization.

We  have  standardizations   that  we  work with the teams

to  standardize  either  across  projects   or  within  a  specific project,

depending  on  the  needs and  for  process  areas.

We  had  this  problem  early  on  that   we weren't getting consistent naming

and  it  was  causing  problems  and  rework.

Now,  we  have  this  standardization  put  in  place.

Also,  the  reporting  decimals   that  we  need to use,

the  minimum  recorded  decimals, what  names  we  use  when  we  write  this

in  a  report,  our  unit,  and  then a  default  transform  to  apply.

That's  our  attribute  standardization.

And  then  for  our  group  standardization, it's  very  similar  identifying  columns,

except  we  have  this  additional  feature here  that  we  can  require

only specific levels  be  present and  otherwise  will  be  flagged.

We  can  also  require  that they have  a  specific  value  ordering.

So  let,  for  example, the process steps  are  always  listed in process step order,

which  is  what  we  need  in  all  our  output.

Okay,  so  I'm  going  to  show an  example  of  this.

Let  me  see  if  I  can  close

some of the  previous  stuff   that  we  have  open  here.

Okay,  so  let  me  go  to  this  example.

So  here's  the  file. The  data  has  been  compiled.

We  think  it's  ready  to  go,  but  we're going  to  make  sure  it's  ready  to  go.

So  we're  going  to  do   the  standardization.

First  thing  is  looking  at  attribute

that  recognizes  the  attributes that  are  erred the  standard  names.

And  then  what's  left  is  up  here.

And  we  can  see  immediately these are process parameters,

which we don't have   the  standardization  set up  for.

But  we  see  immediately that  something's  wrong  with  this,

and  we  see  in  the  list  of  possible attributes  that  the  units  are  missing.

We  can  correct  that  very  easily.

It  will  tell  us  what  it's  doing, we're  going  to  make  that  change.

And  then  it  generates  a  report,

and  then  that  stoplight  coloring and  says,  oh,  we  found  these.

This  is  a  change  we  made, pay  attention  to  this  caution.

These  are  ones  we  didn't  find.

And  this  report  is  saved back  to the data table

so  it  can  be  reproduced   on  demand.

And  I'll  go  through   the  group standardization

just  to  take a  quick  look  at  that.

Here,  it's  telling  me  red, stop  light  coloring.

We  have  a  problem  here, you're  missing  these  columns.

The  team  has  required  that this information  be  included.

It's  going  to  force  those columns  onto  the  file.

We  have  the  option  with  the  yellow to  add  additional  columns.

And  so  we'll  go  ahead  and  run  that, and it's  telling  us  what  it's  going  to  do.

And  then  it  does  the  same  thing,   creates  a  report.

And  we  look  through  the  report and we notice  something's  going  on  here.

Process  scale.

Our  process  scale  can  only have  large  lab  micro.

Guess  what,  we  have  a  labbb. We  have  an  extra  B  in  there.

So  that's  an  error. If we  find  that  value,  correct it.

Rerun  the  standardization and  everything  is  good  there.

I  did  want  to  point  out one  more  thing  here.

You'll  see  that   these  are  our  attributes,

there  are  these  little  stars indicating  properties.

The  properties  that  are  assigned   when  we did the standardization

is  this  custom  property  table  deck.

And  that's  going  to  pass  information  to the system

about  what  the  reporting  precision  is   when  it  generates  tables.

Also,  our  default  transformation   for  HCP was logged,

so  it  automatically  created the  log  transform  for  us.

So  we  don't  have  to  do  that.

Okay.

That's  the  standardization,  let's  move  on   to  a much more  interesting  things now.

The  PC  analysis.

Before  I  get  to  that, I  just  want  to  mention  that

we have a  module   for  scaled- down  model  qualification.

And  essentially,  it's  using JMP's built- in  equivalents  testing.

But  it  enhances  it  by  generating   some custom  plots

and summarizing all that  information at a table that's  report  ready.

It's  beautiful.

Unfortunately,  we  don't have  time  to  cover  that.

I'm  going  to  go  now  into  the  PC  analysis, which  I'm  really  excited  about  here.

I  have  this.

Standardization  has  already   been  done.

We  have  this  file   that  contains  lab results,

experimental  design that's  been  executed  at  labscale.

We  have  large  scale  data  in  here. We can't execute...

It's  not  feasible   to  execute  BOEs  at  large scale,

but  we  have  this at  the  control  point.

We  want  to  be  able  to  project our  models  to  that  large  scale.

And  because  we  have  different  subsets and we have potentially different models,

this  one  only  has  a  single  model,

but we can  have  different  models and  different  setups,

we  decided  to  create  a   BPC-Stat  workflow and  we  have  a  workflow  set up  tool

that  helps  us  build  that  based  on  the particular  model  we're  working  with.

I  can  name  each  of  these  workflows and  I  provide  this  information

that it's going  to  track throughout the  whole  analysis.

What  is  our  large- scale  setting, what  are  our  responses?

Notice  this  is  already populated for me.

It  looked  at  the  file  and  said,  oh, I know these responses,

they're  in  your  standardized  file, they're  in  this  file,  they  exist.

I  assume  you  want  to  analyze this  and  they  get  pre- populated.

It  also  recognize  this   as  a  transform

because  it  knows  that  for  that  HCP, we want that  on  a  log  transform.

And it's going to  do  it   internally  transform,

which  means  JMP   will  automatically  back transform it

so  that  scientists  can  interpret  it   on  the  original  scale.

There  are  some  additional  options  here. This  PC  block  right  now  is  fixed.

In  some  cases,  the  scientists  want   to look at  the  actual  PC  Block  means.

But  for  the  simulation,  we're  interested in  a  population- type  simulation.

We  don't  want  to  look  at  specific  blocks, we  want  to  see  what  the  variability  is.

So  we're  going  to  change   that PC Block factor

into  a  random  effect when  we  get  to  the  simulation.

And  we're  going  to  add   a  process  scale to our models

so  we  can  extend our  model  to  a  large  scale.

The  system  will  review  the  different process  parameters  and  check  the  coding.

If  there's  some  issues  here or a coding missing,

it will automatically flag that   with  the  stoplight  coloring.

We  have  here  the  set  point. Very  tedious  exercise,  annoying.

We  constantly  want  to  show  everything at the set point in the profilers

because  that's  our  control,

not  the  default  that  JMP's calculating  the  mathematical  center.

So  we  built  this  information  in   so that it could  be  automatically  added.

And then we can  define  the  subsets  for  an  analysis.

And  for  that,  we  use  a  data  filter.

I'll show here  for  this  data  filter

and  there's  explanation  of  this   in  the  dialogue.

But  we  want  to  do  summary  statistics   on  a  small  scale.

So  I  go  ahead  and  select  that.

It  gives  feedback  on   how  many  rows  are in that subset

and  what  the  subset  is  so we can double- check  that  that's  correct.

And  then  for  the  PC  analysis,   in  this  case,

I have the model setup  so  that,  of  course,

it's going to analyze the DOE with  the  center  point

but  it's  also  going to do  this single factor study,

or  what  they  call  OFAT   and  SDM  block,

separate  center  points that  were  done  as  another  block.

And  that's  all  built  into  another block  in  the  study  for  that  PC  block.

Lastly,  I  can  specify  subsets for confirmation points,

which  they  like  to  call   verification points,

to  check and  to  see   how  well  the  model  is  predicting.

We  don't  have  those  in  this  case.

And  for  what  is  our  subset   for  large scale,

that  would  include  both  the  lab   and  the  large  scale  data.

Since  it's  all  the  data  in  this  case, I  don't  have  to  specify  any  subset.

Now,  I  have  defined  my  workflow.

I  click  okay,  and  it  saves  all that  information  right  here  as  a  script.

If  I  right- click  on  that,  edit  it, you  can  see  what  it's  doing.

It's  creating  a  new  namespace.

It's  got  the  model  in  there, it's got all my responses,

and  everything I  could  need  for  this  analysis.

A s  soon  as  you  see  this, you  start  thinking,  well,

if  I  have  to  add  another  response, I  can  stick  another  response  in  here.

But  that  violates  the  principle of  no  script  editing.

Well,  sometimes  we  do  it, but  don't  tell  anybody.

What  we  did  is  we  built  a  tool   that  has a workflow editor

that  allows  us  to  go  ahead   back  into  that  workflow

through point  and  click  and  change  some of  its  properties  and  change  the  workflow.

I'm  going  to  go  ahead  now  and  do  the  analysis.

And  this  is  where the  magic  really  comes  in.

When  I  do  the  PC  analysis  set up,

it's  going  to  go  take  that   workflow  information  and  apply  it

across the entire set  of  scripts that we need  for  our  analysis.

And  you  see  what  it  just  did  there.

It  dropped  a  whole  bunch  of  scripts. It  grouped  them  for  us.

Everything  is  ready  to  go.

It's  a  step- by- step  process, the  scientists  can  follow  it through.

If  there  are  scripts   that  are  not applicable,

they  can  remove  those  scripts  and  they're  just  gone.

We  don't  worry  about  them.

And  then  for  the  scripts  that  are  present, we  have  additional  customization.

These  are  essentially generator  scripts.

And  you  can  see   it  generates  a  dialogue

that's  already  pre- populated   with  what  we should need,

but  we  have  additional  flexibility   if  we  need  it.

And  then  we  can  get  our  report and we can enhance  it  as  we  need  to,

in  this  case, subsets  I  may  want  to  include.

And  then  resave  the  script  back  to  the table  and  replace  the  generator  script.

Now,  I  have  a  rendered  script  here that  I  can  use  that's  portable.

Then  for  the  PC  analysis, we  have  data  plots.

Of  course,   we  want  to  show  our  data.

Always  look  at  your  data, generate  the  plots.

There's  a  default  plot  that's  built.

And  now  the  user, we only did one plot

because  we  wanted  the  user   to  have  the  option  to  change things

so  they  might  go  in  here, say,  get  rid  of  that  title.

I  just  changed  the  size   and  I  add  a  legend,  whatever.

You  can  change  the  entire  plot if  they  wanted  to.

And  then  one  of  their   all- time  favorite features of BPC-Stat

seems  to  be  this  repeat  analysis.

Once  we  have  an  analysis  we  like, we  can  repeat  it.

And  what  this  is  doing  is  it's  hacking the column switcher

and  adding  some  extra  features  onto  it.

It'll  take  the  output, dump  it  in  a  vertical  list  box  or  tab  box,

and  allow  us  to  apply  our  filtering either  globally  or  to  each  analysis.

Now,  I'm  in  the  column  switcher

and I can tell  it  what  columns   I  want  it  to  do.

This  works  for  any  analysis, not  just  plotting.

Click  OK.

It  runs  through  the  switcher, generates  the  report.

There  I  have  it. All  my  responses  are  plotted.

That  was  easy.

I  go  down  and  there's  the  script  that  recreates  that.

I  can  drop  it  here, get  rid  of  the  previous  one.

Done.

Descriptive  statistics. Here  we  go.

It's  already  done.

I  have  the  subsetting  applied, to  have  the  tables  I  need.

Look  at  this.

It's  already  formatted   to  the  number of decimals I needed

because  it's  taking  advantage   of  those  properties  that  we  had assigned,

those  unique  properties based  on  this  table  standardization.

So  that  one  is  done.

And  then  the  full  model.

Full  model  is  set  up,  ready  to  go for what would  you  think?

It's  ready  to  go   for  residual  assessment.

We  can  go  through  each   of  the  models one at a time

and  take  a  look  to  see  how  the  points  are  behaving,  the  lack  of  fit.

Does  it  look  okay?

Here,  we  have  one  point  that  may  be a little  bit  off  we  might  want  to  explore.

Auto  recalc  is  already  turned  on,

so  I  can  do  a  row  exclude and it will  automatically  update  this.

Or  we  have  a  tool  that  will exclude  the data point in a new column

so  that  I  can  analyze  it  side  by  side.

And  then  since  I've  already   specified my responses,

in  order  to  include  that  side  by  side,

I  would  have  to  go  back and  modify  my  workflow.

And  we  have  that   workflow  editor  to  do  that.

I'm  just  going  to  skip  ahead  to  save some  time  where  I've  already  done  that.

This  is  the  same  file,   same  analysis,

but  I've  added  an  additional  response and  it's  right  here.

Yield  without  run  21.

Now,  scientists  can  look  at  this   side  by side  and  say,

you  know  what,  that  point, yeah,  it's a little bit unusual statistically,

but  practically there's  really  no  impact.

All  right,  let's  take  this  further.

This  is  our  routine  process.

We  do  take  it  all  the  way   through the reduced model

because  we  want  to  see   if  it  impacts  model  selection.

We  have  automated  the  model  selection

and  it  takes  advantage   of  the  existing stepwise for forward AIC

or  the  existing  effects  table  where   you  can  click  to  remove by backward

selection manually if you want, this  automates  the  backward  selection

which  we  typically  use for  split  pot  designs.

We  also  have  a  forward  selection for mixed models  which  is  not  currently

a JMP feature and  JMP that we find  highly  useful.

I'm  going  to  go  ahead,  since  it's  a  fixed  model,

I'm going to do that  and gets the  workflow  information.

I  know  I  need  to  do  this   on  the  full  model.

It  goes  ahead  and  does  the  selection.

What  it's  doing  in  the  background  here is  it's  running  each  model,

it's  constructing  report   of  the  selection that it's done

in  case  we  want  to  report  that.

And  it's  going  to  save  those  scripts back  to  the  data  table.

There's that  report  right  there that contains all  the  selection  process.

Those  scripts  were  just generated  and  dumped  back  here.

Now,  I  can  move  those  scripts back  into  my  workflow  section.

I  know  the  reduced  model  goes in there

and  this  is  my  model selection  step  history.

I  can  put  that  information  in  there.

Okay,  so  this  is  great.

Now,  when  I  looked  at  my  reduced  model, I  have  that  gone  through  the  selection.

Now  I  can  see  the  impact  of  the  selection on  removing  this  extra  point  here.

And  again,  we  see   there's  just  basically,

likely the scientists would conclude there's  just  no  practical  difference  here.

And  they  could  even  go,  and  should  go

and  look  at  the  interaction  profilers as  well,  compare  them  side  by  side.

This  is  great.

We  want  to  keep  this  script  because  we want to keep track

of  the  decisions  that  we  made, so  there's  a  record  of  that.

But  we  also  want   to  report  the  final  model.

So  we  want  a  nice,  clean  report.

We  don't  want  that  without   running response in there

because  we've  decided   that  it's  not  relevant,

we  need  to  keep  all  the  data.

Another  favorite  tool  that   we developed is  the  Split  Fit  Group

which  allows  us to break up  these fit groups

We have the reduced model  here. Take  the  reduced  model.

And allows us to  break  them  up   into as many  groups  as  we  want.

In  this  case,  we're  only  going   to  group  it up into one group

because  we're  going   to  eliminate  one  response.

We  want  one  group.

When  we're  done,  we're  just  using  this to eliminate this response

we  no  longer  want  in  there.

Click Okay.

That's  some  feedback  from   the model fitting,  and  boom,  we  have  it.

The  fit  group  is  now  there  and  without response  analysis  has  been  removed.

Now  we  have  this  ready  to  report.

Notice  that  the  settings  for  the  profiler, they're  all  the  settings  we  specified.

It's  not  at  the  average  point or the  center  point,

it's  at  the  process  set point,

which  is  where  we  need  it to  be  for  comparison  purposes.

It's  all  ready  to  go.

Okay,  so  that  generates   that script there  when  I  did  that  split.

I  can  put  it  up  here and  I  can  just  rename  that.

That's  final  models.

Okay,  very  good.

Now,  for  some  real  fun.

Remember  we  had  talked  about   how tedious it is

to  set up  this  simulation  script.

Now  watch  this. Watch  how  easy  this  is  for  the  scientists.

And  before  I  do  this, I  want  to  point  out  that  this  was,

of  course,  created  by  a  script,  obviously JSL,

but  this  is  a  script  that  creates a script that  creates  a  script.

So  this  was  quite  a  challenge for  Adsurgo  to  develop.

But  when  I  run  this, I can pick my model here,

final  models,   and  then  in  just  a  matter of seconds,

it  generates  the  simulation  script   that  I  need.

I  run  that,  and  boom. There  it  is,  all  done.

It set up  the  ranges  that  I  need for  the  process  parameters.

It's  set  them  up  to  the  correct  intervals. It  set  the  ad  random  noise.

But  there's  even  more  going on  here  than  what  appears.

Notice  that  the  process  scale   has  been  added,

we  didn't  have  that  in  the  model  before.

That  was  something  that  was  added so that  we  could  take  these  labs,

scale  DOE  models,   and  extend  them  to  the  large  scale.

Now  we're  predicting  large  scales. That's  important.

That  was  a  modification to  the  model.

Previously, very  tedious  editing   of the script was  required  to  do  that.

Notice  that  we  also  have  this  PC  block random  effect  in  here  we  had  specified

because  we  don't  want to  simulate  specific  blocks,

now  it's  an  additional  random  effect.

And  the  total  variance  is  being  plugged into  the  standard  deviation

for the ad random  noise,   not  the  default  residual  random  noise.

We  also  added  this  little  set  seed  here so  we  can  reproduce  our  analysis  exactly.

So  this  is  really  great.

And  again,  notice  that  we're  at  the process  set  point  where  it  should  be.

Okay,  last  thing  I  want to  show  here  is  the  reporting.

We essentially  completed  the entire analysis,

you  can  see  it's  very  fast.

We  want  to  report  these  final  models  out   into  a  statistics  report.

And  so  we  have  a  tool  to  do  that.

And  this  report  starts with  a  descriptive  statistics.

I'm  going  to  run  that  first,

and  then  we're  going  to  go  and  build the  report,  export  stats  to  Word.

And  then  I  have  to  tell  it which  models  do  I  want  to  export.

It's  asking  about  verification  plots.

We  didn't  have  any  in  this  case for  confirmation  points.

So  we're  going  to  skip  that.

And  then  it  defaults  to  the  default directory  that  we  set  the  output.

I'm  going  to  open the  report  when  I'm  done.

And  this  is  important.

We're  leaving  the  journal  open for  saving  and  modification.

Because  as  everybody  knows, when you copy  stuff,

you  generate  your  profilers,

you  dump  them  in  Word,   and  there's  some  clipping  going  on.

We  may  have  to  resize  things,

we  may  have  to  put something  on  a  log  scale.

We  can  do  all  that  in  the  journal and  then  just  resave  it  back  to  Word.

That  saves  a  step. So  we  generate  that.

I  click  okay  here.

It's  reading  the  different  tables   and  the  different profilers,

and  it's  generating this  journal  up  here.

That's  actually  the  report that  it's  going  to  render  in  Word.

And  it  will  be  done   in  just  a  second  here.

Okay.

And  then  just  opening  up  Word.

And  boom,  there's  our  report.

So  look  at  what  it  did.

It  puts  captions  here,  it put  our  table.

It's  already  formatted   to  the  reporting  precision  that  we  need.

It  has  this  footnote  that  it  added,  meeting  our  standards.

And  then  for  each  response, it  has  a  section.

And  then  the  section  has  the  tables with  their  captions,  and  the  profilers,

and  interaction  profiler, footnotes,  et  cetera.

And  it  repeats  on  and  on  for  each  attribute  over  and  over.

It  also  generates  some  initial  text

that  the  scientists  can  update with  some  summary  statistics.

And  so  it's  pretty  much  ready  to  go and  highly  standardized.

That  completes  the  demo   of  the  system.

Now,  I  just  have  one  concluding  slide that  I  want  to  go  back  to  here.

So,  in  conclusion,   BPC-Stat, it's  added  value  to  our  business.

It's  enabled  our  process  teams. It's  paralyzed  the  work.

It's  accelerated  our  timelines.

We've  implemented  a  standardized, yet  flexible  systematic  approach

with  that  higher,  faster  acceleration, and  much  more  engagement.

Thank  you  very  much.