Addressing Large, Complex, Unstructured Statistical Problems
Roger W. Hoerl, Manager, Applied Statistics Lab, GE Global Research
Ronald D. Snee, President, Snee Associates
This session will discuss how to attack large, complex, unstructured statistical problems. It is based on the second edition of the book Statistical Thinking: Improving Business Performance (Hoerl and Snee 2012), recently released as part of the Wiley and SAS Business Series. There is considerable interest in determining how to attack complex statistical problems, i.e., those that "…do not correspond to a recognizable textbook chapter" (Meng 2009). Most textbook problems, as well as problems discussed at conferences and in publications, tend to be narrowly defined and require one "correct" statistical method to solve them. In contrast, large, complex problems typically are broad in scope and require some upfront work simply to provide structure. They require a dynamic process view, as opposed to static population view, for proper understanding and interpreting multiple data sources. In addition, such problems almost always require effective integration of multiple statistical methods into an overall approach to scientific inquiry, not just identification of the one "correct" statistical tool. In short, they require creative and innovative thought statistical thinking in addition to knowledge of the tools. The authors have used the term statistical engineering to refer to the integration of multiple statistical methods and tools to address complex problems (Hoerl and Snee 2010). JMP can provide significant insight to practitioners faced with such problems. While no easy method exists for large, complex problems, the authors present guidelines for thinking about and attacking them.