Okay,  welcome  everybody  to  this  presentation  in  which  I  will  talk  about  Pearl's  Influenza  Studies.  Pearl  a biostatistician,  he  lived  quite  a  while  ago  during,  specifically,  the   Spanish Flu,  and  he  was  the  first  one  to  analyze  the  weekly  data  that  was  collected  by  the  US  Bureau  of  the  Census  about  the   Spanish Flu,  which  occurred  in  1918,  1920,  and  this  for  the  large  American  cities.

Pearl,  he  was  the  first  to  do  so  to  analyze  the  data  regarding  the   Spanish Flu,  and  he  wrote  two  reports  about  it,  his  influenza  studies  one  and  two,  which  we  will  revisit  now  during  this  presentation,  and  we'll  see  how  we  will  be  able  to  look  into  the  data,  analyze  the  data  using  JMP.  This  is  joint  work  with  Chris  Gotwalt,  who  contributed  to  the  methodological  component,  and  Guido  Erreygers,  who  initiated  the  idea  of  revisiting  Pearl's  influenza  studies.

The  overview  of  this  talk  is  as  follows.  First,  I'll  discuss  a  little  bit  the   Spanish Flu,  what  it  was  all  about.  Quite  a  deadly  pandemic  at  that  time.  Then  I'll  introduce  Pearl's  Influenza  Studies,  and  I'll  talk  a  little  bit  about  the  data  that  Pearl  used  in  his  analysis.  We  have  added  some  census  data  from  1910  on  top  of  that  for  our  analysis,  which  consists  of  a  variable  selection  procedure  with  a  null  factor,  a  random  factor,  an  independent  normal  factor  in  our  analysis,  and  bootstrap  simulation  from  the  data.  Then  we'll  be  discussing  our  results  and  compare  those  results  with  Pearl's  results,  and  then  we'll  conclude.

First  of  all,  the   Spanish Flu  pandemic,  to  frame  that  a  little  bit,  it  was  one  of  the  deadliest  pandemics  in  history,  as  witnessed  by  this  list  here  showing  the  world's  deadliest  pandemics  in  overtime.   You  can  see  that  the   Spanish Flu  here  ranks  fifth  in  terms  of  the  number  of  deaths  that  it  caused.  T his  list  is  headed  by  the  Black  Deaths,  which  killed  about  four  times  more  people  than  the   Spanish Flu  did.   Then  below  this  list,  you  see  COVID- 19  pandemic  appearing,  which  we  got  all  exposed  to.   It  also  appears  in  this  list  still.

Just  as  with  COVID- 19,   gatherings  were  more  encouraged  to  happen  outside,  and  also  at  the  times  already  of  the   Spanish Flu  pandemic  there,  that  was  the  case  as  well,  and  even  for  classes  to  happen  outdoors,  as  you  can  see  on  this  photo.

The   Spanish Flu  pandemic  consisted  of  three  waves.  The  first  one  started  in  March  1918  in  the  US  and  spread  to  Europe  and to  the  rest  of  the  world.  Then  a  more  severe  wave,  the  severest  one  started  in  August  1918  in  France  and  spread  rapidly  to  the  rest  of  the  world,  as  well  as  coincided  with  the  end  of  the  First  World  War.  Then  a  third  one  which  was  less  severe  than  the  second  one,  but  more  severe  than  the  initial  wave,  started  at  the  early  beginning  of  1919  and  hit  some  specific  countries  like  Australia  and  Great  Britain.

Here  you  see  the  timeline  of  the  three  waves  of  the   Spanish Flu  pandemic  occurring  in  the  US,  more  specifically  because  we  have  Pearl's  data  from  the  US  which  we  look  into.  The  death  toll  was  humongous  and  most  deaths  occurred  actually  in  India  and  China.

What  was  specific about  the   Spanish Flu  pandemic  was  that  many  young  healthy  adults  died  as  shown  by  a  W  pattern  of  mortality  rates,  which  is  here  shown  by  the  full  black  line.   In  contrast  to  the  U  shape  of  the  normal  seasonal  influenza  and  pneumonia  mortality  rates  which  were  registered  prior  to  the  pandemic.

For  seasonal  influenza  and  pneumonia,  this  U  shape  shows  that  individuals  younger  than  four  years  of  age  and  people  older  than  75  years,  that  these  were  most  hit  by  seasonal  influenza  and  pneumonia.   Characteristic  of  the   Spanish Flu  pandemic  was  that  besides  those  two  age  groups,  also  the  young  adults  in  the  range  between  20  and  40  years  of  age  got  specifically  hit  by  this  pandemic,  eventually  leading  up  to  a  huge  death  toll.

Then  further  onwards  throughout  history,  epidemiologists  and  historians  of  epidemiology  have  applied  statistical  analysis  to  the   Spanish Flu  pandemic.  More  specifically,  people  worked  on  the  US  as  well,  namely,  Markel  and  colleagues,  they  studied  the  effect  of  non- pharmaceutical  interventions  in  American  cities  like  school  closure,  cancelations  of  public  gatherings.

Mamelund  studied  the  influence  of  geographical  isolation  of  specific  areas  and  specific  regions  in  the  US.   Clay,  Lewis,  and  Severnini  studied  cross- city  variation  of  438  US  cities,  but  also  elsewhere  around  the  world,  so  not  only  in  the  US  but  also  in  Europe,  like  in  Norway  and  Spain.  Data  about  the   Spanish Flu  pandemic were  analyzed  together  with  census  data  and  so  forth.   The   Spanish Flu  pandemic  kept be  intriguing  for  many  researchers  over  time.

About  Pearl's  influenza  studies  now,  here  you  see  the  first  of  his  influenza  study,  the  beginning  of  it.   First,  he  talks  about  the  severity  of  the  outbreak  and  actually  that  the  death  toll  for  the  United  States  was  set  at  about  550,000,  which  is  about  five  times  more  than  the  number  of  people  who  actually  died  during  the  First  World  War.  Hence,  showing  the  severity  of  this  pandemic  and  actually  how  deadly  it  was,  how  explosive  it  was.  That  was  specifically  what  Pearl  was  interested  in  examining  and  relating  to  other  variables.

The  second  of  his  influenza  studies  looks  as  follows,  so  the  beginning  at  least.   That's  an  update  of  the  first  of  his  studies  because  he  got  some  criticism  after  first  study  by  some  peers  who  criticized  him  for  not  being  accurate  with  certain  variables.  He  had  issues  with  what  we  now  refer  to  as  construct  validity.  Some  of  the  data  were  not  really  measuring  what  they  were  supposed  to  measure,  so  they  were  not  so  accurate  and  they  could  be  defined  more  appropriate,  more  accurate  in  order  that  they  really  measure  what  they  should  measure.   He  did  that.  He  tackled  the  data  again,  the  variables.  Set  up  new  definitions  of  the  variables  and  moved  forward.

The  data  for  the  first  of  his  studies  looks  as  follows.  We  have  data  from  Pearl  of  39  American  cities.   The  response  variable  he  wanted  to  study  was  the  epidemicity  index,  which  is  given  by  the  symbol  I 5  in  his  studies.  That  was  the  first  response  variable  he  wanted  to  study  and  he  wanted  to  also  find  other  variables  like  demographics  that  could  be  predictive  of  this  response  variable.

Now,  this  response  variable  was  a  measure  for  the  explosiveness  of  the  outbreak  in  terms  of  epidemic  mortality,  so  how  explosive  the  outbreak  was  and  hitting  the  various  cities  in  the  US.   He  defined  the  peak  time  ratio,  so  the  peak  of  the  excess  mortality  rates  divided  by  the  peak  date.   In  this  way,  he  wanted  to  compare  cities  with  one  single  very  sharp  peak  to  cities  with  a  long  flattened  curve  of  excess  mortality  rates.   He  devised  this  epidemicity  index  himself.

He  wanted  to  relate  this  epidemicity  index  to  various  factors  like  the  demographics.  First  of  all,  the  population  density  for  1916,  and  then  the  geographical  position,  which  is  a  straight  line  distance  from  Boston.  He  also  included  the  age  distribution,  chi- square,  which  is  an  age  constitution  index  showing  the  deviation  in  the  age  distribution  of  the  cities  from  a  fixed  standard  age  distribution.  He  also  studied  the  percentage  population  growth  in  the  decade  1900,  1910.

Besides  the  demographics,  he  also  involved  the  death  rates  for  1916,  so  prior  to  the  pandemic.  First  of  all,  a n  aggregate  death  rate  from  all  causes  and  then specific diseases,   death  rates  for  specific  diseases,  namely  pulmonary  tuberculosis,  organic  heart  disease,  acute  nephritis  and  Bright's  disease,  or  failure  of  the  kidneys,  influenza,  pneumonia,  typhoid  fever,  cancer,  and  measles.  There  was  quite  some  correlation  between  these  death  rates.

I already  told  you  a  little  bit  about  the  response  variable  that Pearl  developed,  namely  the  epidemicity  index.  That  did  not  arise  into  just  a  single  attempt,  but  he  started  actually  from  I 1.   First  of  an  epidemicity  index  that  he  further  improved  into  I 2,  I 3,  I 4,  and  then  he  was  happy  enough  to  move  along  and  to  work  with  the  I 5  epidemic ity  index.   To  distinguish  then  cities  with  single,  very  sharp  peaks  to  those  with  long,  low  flat  curves  of  epidemic  mortality.

He  updated  the  variables  from  his  first  study  and  the  second  study,  and  in  that  sense,  he  also  updated,  he  also  modified  the  epidemicity  index  and  the  new  epidemicity  index.  He  referred  to  that  one  as  I 6.   As  another  variable  of  interest,  he  also  defined  to  this  destructiveness  variable  as  containing  excess  mortality  rates.  Then  he  wanted  to  relate  these  two  responses  in  his  second  study  with  the  normal  death  rates,  which  were  the  mean  death  rates  over  the  three  years,  1915  to  1917.  Then  he  also  brought  in  the  demographics.  Again,  he  modified  the  age  constitution  index  based  on  the  1910  census  data,  and  he  used  that.

Then  also  he  involved  the  sex  ratio  of  the  population  of  males  versus  females.  The  population  density  of  1910  is  used  in  the  first  of  his  studies,  remained  in  the  second  study.  Then  instead  of  a  geographical  position,  he  used  latitude  and  longitude  in  his  second  study.  Then  lastly,  he  used  the  percentage  population  growth  in  the  decade  1900,  1910,  again.

These  were  Pearl's  data  to  which  we  added  some  additional  census  data  from  1910  which  are  given  over  here.  Instead  of  the  age  constitution  index,  we  used  the  share  of   the  different  age  groups,  and  the  pure  numbers  actually,  because  we  were  not  really  happy  with  the  way  Pearl  defined  the  age  constitution  index.  It  was  really  quite  complex  and  not  clear  enough  so  that  we  thought  to  just  go  about  and  use  the  1910  share  ages  instead  of  this  age  constitution  index  developed  by  Pearl.

Besides  that,  we  looked  into  the  number  of  persons  to  a  dwelling,  the  percentage  of  homes  owned,  the  school  attendance  of  population,  6  to  20  years  of  age,  and  the  illiteracy  in  the  population,  10  years  of  age  and  over,  and  see  whether  one  of  these  factors  or  some  of  these  factors  could  be  predictive  of the  tree  response  variables  in  Pearl's  studies.

What  was  Pearl's  analysis  all  about?  Well,  he  got  into  multiple  correlation.  He  studied  all  the  data  making  use  of  partial  correlation  coefficients  as  well  as  the  normal  correlation  coefficients  of  zero  order,  but  he  did  that  very  rigorously.  He  had  computed  this  all  by  hand  and  did  this  quite  well  actually  and  took  into  account  various  other  factors  in  these  partial  correlations  by  having  those  other  variables  being  constant.   For  the  partial  correlations  and  the  other  correlations,  he  also  computed  probable  errors  to  find  out  whether  these  correlation  coefficients  were  significant  or  not,  so  he  did  this  quite  well.

Now,  the  analysis  that  we  are  using  is  the  one  in  which  we  are  going  to  actually  select  variables  with  a  null  factor  and  we  are  going  to  bootstrap  our  data.  We  are  doing  so  because  our  P  values  for  our  data,  which  are  unfortunately  not  orthogonal,  they  can   become  heavily  biased.  Since  we  are  not  really  using  nicely  orthogonal  data,  the  P  values  can  become  quite  biased  towards  zero.  It's  always  the  danger  with  P  values  for  observational  data  that  unimportant  factors  all  of  a  sudden  become  important  and  that  the  type  one  error  rate  is  not  under  control.

To   solve  the  fact  that  the  P  values  are  not  uniformly  distributed  anymore  in  the  case  of  an  unimportant  variable,  to  solve  that  issue,  we  are  going  to  involve  a  random  variable,  a  null  factor,  an  independent  normal  variable  into  the  analysis  and  see  to  what  extent  it  appears  in  our  procedure  of  selecting  variables.  This  idea  is  inspired  by  or  originated  from  the  JASA  paper  by  Wu,  Boos,  and  Stefanski  in  2007.

Specifically,  what  we  have  done  is,  well,  we  included  a  single  null  factor  in  the  variable  selection  procedure  and  performed  2,500  bootstrap  replicates  for  variable  selection  using  JMP.   Then  we  calculated  the  proportion  of  times  each  variable  enters  the  model.  Variables  that  enter  as  often  or  less  than  the  null  factor  are  ignorable,  and  variables  that  enter  more  often  than  the  null  factor,  well,  these  are  the  ones  that  we  are  going  to  select  as  being  predictive  of  the  response  variable.

In  JMP,  we  specified  two  new  columns,  and  actually,  one  formula  is  needed.  That's  the  formula  for  the  bootstrap  frequency  column  that  you  see  here.  The  bootstrap  frequency  column  is  based  on  our  null  factor,  which  is  an  independent  normal  variable  that's  reinitialized  each  time  during  the  bootstrap  simulation.   Based  on  this  reinitialization,  the  frequency  column  gets  updated  so  that  we  have  a  100 %  resampling  of   our  sample size of our data  with  the  fixed  sample  size  as  it  is.

Then  for  the  variable  selection,  what  kind of regression  also  would  we  apply  that  we  could  find  out  in  the  generalized  linear  model  platform  of  JMP,  because  taking  into  account  the  frequency  column  also,  if  we  do  variance  selection,  we  also  need  to  do  the  distribution  determination  at  the  same  time.  It  has  to  happen  simultaneously.  You  can't  do  it  apart  based  on  the  original  data.   Actually,  the  graph  on  the  left- hand  side  is  a  little  bit  misleading  because  that  graph  contains  the  original  data,  but  distribution  determination  you  should  actually  do  while  doing  the  variable  selection  itself.

Assuming  a  Poisson  distribution,  well,  that  assumption  was  not  rejected,  so  we  could  actually  move  forward  with  the  assumption  of  a  Poisson  distribution,  which  is  also  maintained  in  the  literature  throughout  for  mortality  rates  and  so  forth  for  analysis.  It  was  not  really  rejected  and  it  was  good  to  assume  such   a  Poisson  regression  for  the  analysis.

Then  also  we  applied  Poisson  regression  in  combination  with  variable   selection  for  the  epidemicity  index  I6.  However,  we  switched  to  normal  regression  or  less  regression  for  the  third  response  variable,  the  destructiveness  variable.

The  way  to  apply  this  regression  in  JMP  by  means  of  variable  selection  is  by  using  the  generalized  regression  platform  where  we  then  define  the  response  variable  for  analysis  and  put  the  frequency  column  into  the  freq  box.   Then  as  model  terms  in  the  construct  model  effects  window,  we  involve  all  the  variables,  all  Pea rl's  variables,  together  also  with  our  null  factor  or  independent  normal  variable  that  we  also  included.

Then  we  move  forward  with  the  regression  procedure  by  selecting  forward  selection  estimation  method.  As  a  criterion,  we  used  the  Akaike  information  criteria  with  the  correction  for  small  sample  sizes  to  decide  upon  the  final  model  to  include  the  final  model  then  to  select  with  the  selection  of  the  variables.

The  solution  part  that  you  see  here  is  based  on  normalized  data  to  put  them  on  the  same  scales, so scales  and  center  data  and  also  to  diminish  the  effect  of  multicollinearity  in  the  data  because  there's  quite  some.   Then  you  see  again,  the  original  predictors  popping  up  in  the  lower  output.   Then  we  select  a  model  with  the  lowest  Akaike  information  criteria.  One  after  the  other,  the  variables  coming  being  selected  by  forwards  variable  selection  based  on  the  Akaike  information  criteria.  As  you  can  see  in  this  output  actually  here,  the  null  factor  got  into  the  model.  The  null  factor  completely  unimportant,  uninformative,  so  got  into  the  model,  which  is  not  good,  of  course.

We  had  to  do  this  estimation  method,  this  variable  selection  procedure  2,500  times,  and  we  did  this  in  JMP  by  right  clicking  on  the  estimate  column  and  then  hitting  simulate  and  then  selecting  the  frequency  column  as  a  column  to  switch  in  and  out   between  the  different  bootstrap  replicates  so  that  initializations  of  the  null  factor  were  always  being  guaranteed  between  the  different  bootstrap  replicates.  Then  finally,  so we  got  the  2,500  model  selections  out  of  the  bootstrap  simulation.

Then  we  computed  the  proportion  that  all  of  these  variables  got  into  the  model  or  were  given  as  non- zero  estimates.  We  were  represented  by  non- zero  estimates.  Especially,  of  course,  we  were  interested  in  how  many  times  the  null  factor  appeared  in  the  selection  of  the  models.  This  turns  out  to  be  41 %  of  the  times,  which  is  quite  high. T hese are  our  new  false  entry  rates,  actually.   A  very  high  percentage  in  which  the  null  factor  got  selected.

We  accounted  for  some  upper  bound,  an  upper  99.9 %  confidence  limit  also  that  we  took  into  account,  and  even  a  little  bit  higher  sometimes.  To  be  assured  that  the  factors  that  you  see  in  green  that  they  appeared  most  often  then  in  the  model  selection.  As  you  will  see  also  another  example  now  too,  where  we  got  tied  with  the  null  factor  as  well.  The  factors  or  the  variables  in  reds,  they  have  not  been  selected  since  their  occurrence  is  lower  than  the  occurrence  of  the  null  factor  over  the  different  bootstrap  simulations.  The  variables  of  interest  here  that  got  selected  and  are  predictive  of  the  epidemicity  index  I5  are  the  death  rates  of  causes,  death  rate  from  organic  heart  disease,  the  death  rates  from  pneumonia,  death  rate  from  cancer,  death  rate  from  measles,  and  the  geographical  position.

Now,  the  death  rate  from  all  causes,  that's  an  aggregate  for  the  death  rates  from  the  individual- specific  chronic  diseases.   We  were  also  interested  to  see  what  would  happen  if  we  were  to  take  this  out  of  the  analysis.   This  is  what  we  did  on  the  following  slides.   We  took  it  out,  death  rate  all  causes,  and  then  we got  a  different  picture.  The  death  rate  from  measles  and  the  death  rate  from  cancer  turned  out  to  be  unimportant  now,  whereas  death  rate  from  pneumonia  got  in  the  green  zone,  as  well  as  the  death  rate  from  pulmonary  tuberculosis.   These  were  also  quite  highly  correlated  with  the  death  rate  of  all  causes.  Death  rate  of  all  causes  masked.  These  variables,  although  some  of  the  death  rates  are  also  correlated  among  each  other,  so  we  have  to  be  careful  here.   That  was  a  new  result  that  we  saw.

Now,  eventually,  we  repeated  these  variable  selections  further  onwards  with  only  the  variables  in  green  that  you  see  here  on  the  screen,  so  the  ones  that  got  selected  to  which  we  added  our  new  1910  census  data,  like  the  shares  of  ages  and  the  illiteracy  and  the  schooling  and  so  forth.  We  did  this  with  and  without  dead  rate  or  causes  to  finally  then  select  the  variables  that  were  common  kinds of  overall  analysis.   All  the  different  analyses  selected  some  other  variables  still,  but  still  there  were  some  variables  which  were  present  all  the  time,  and  these  are  the  ones  that  we  finally  then  contained  into  the  model,  which  is  our  final   Poisson  regression  model  that  we  obtained  in  the  end.

Which  contains  death  rates  from  heart  disease,  organic  heart  disease,  death  rates  from  all  causes  we  kept  telling,  as  also  Pearl  stressed  that  as  being  quite  important.  Share  ages,  0  to  4.  School  attendance  of  the  population,  6  to  20  years  of  age  and  the  geographical  position.   That  was  our  final  regression  model  for  the  first  response,  the  epidemicity  index  I5,  and  below  you  see  the  results  of  Pearl.  Having  done  a  correlation  analysis,  he  was  able  to  identify  pulmonary  tuberculosis,  organic  heart  disease,  and  acute  nephritis  and  Bright's  disease.  Besides  the  death  rate  of  all  causes  that  he  deemed  quite  important,  which  was  actually  also  always  the  case  for  our  analysis.  It  always  came  on  top  of  our  analysis.   We  kept  it  in.

Now,  Pearl  also  pointed  towards  some  specific  chronic  diseases,  but  we  were  not  able  to  put  these  on  top.  We  did  not  find  them  to  be  stably  present.  Overall,  our  analysis,  so  we  did  not  identify  them.  Also,  Pearl,  actually,   after  his  first  study,  got  criticized  because  of  pointing  towards  these  individual  diseases.  Then  in  the  second  of  his  study,  also,  he  was  more  prudent,  more  conservative,  and  only  pointed  towards  the  death  rate  of  all  causes  and  the  organic  diseases  of  the  heart  as  the  ones  that  are  predictive  for  his  modified  index  of  the  explosiveness  of  the  outbreak  I6.

Our  final  analysis  is  the  one  that  you  see  here  pointing  towards  also  death  rates  from  organic  heart  disease,  death  rate  from  all  causes,  the  share  of  the  ages  between  zero  and  four,  and  the  population  density rule.  W e're  always  able  to  see  the  population  density  coming  up  high  in  the  list  of  variables  that  occurred  almost  continuously  over  the  bootstrap  replicates.  That  also  turned  out  to  be  an  important  variable  from  our  analysis.

With  the  destructiveness  variable,  we  did  not  find  all  that  much  of  variation.  The  range  actually  of  values  was  not  as  large  as  for  the  other  two  responses,  the  epidemicity  index .  We  were  only  able  to  identify  a  few  variables  and  specifically  the  death  rates  coming  from  the  organic  heart  disease.  Then  we  identified,  besides  the  share  of  the  youngest  people  between   0 and   4 years,  also  the  share  of  people  ranging  between  20  and  40,  so  25  and  44  years  of  age.  The  healthy  people  actually  prior  to  the  pandemic,  that  was  also  indicative  of  excess  mortality  rates  over  the  different  cities  in  the  United  States.

Pearl's  data  sets,  as  we  could  see  also,  we  only  had  very  little  data.  There  were  very  tiny  39  observations  in  the  first  study  and  in  the  second  study  only  34  observations  because  they  modified  some  of  the  variables,  some  observations  got  lost.   The  data  are  also  observational  with  quite  some  multicollinearity  involved.  The  quality  of  the  data  could  have  been  better.  Therefore,  our  analysis  for  sure  are  useful,  but  they  are  not  magical,  as  well  as  Pearl's  correlation  analysis.  Specifically,  his  first  analysis,  it  was  not  supported.  He  knew  afterwards  that  people  did  not  really  support  his  first  analysis.   Anyway,  as  George  Box  said,  "All  models  are  wrong,  some  are  useful."

We  were  able  to  select  satisfactory  models  in  a  sequential  manner.  First  of  all,  we  included  Pearl's  variables  and  retained  the  selected  variables,  to  which  then  we  added  new  1910  census  variables  to  finally  select  those  variables  that  are  informative  and  each  time  with  and  without  the  death  rate  from  all  causes.  Then  we  retained  the  variables  that  popped  up  in  the  green  zone  to  be  quite  predictive  of  the  response  that  popped  up  all  the  time, so   over  all  the  analysis  that  we  did  to  then  have  the  models  that  I  presented.   I  hope  this  was  informative  for  you  to  listen  to  and  many  thanks.