Subscribe Bookmark



Mar 30, 2012

Experiment design – and how!

Experiments for most of us are demonstrations of scientific principles. We recall the science class where we put litmus paper into a beaker of lemon juice and watched it turn pink.

In scientific research, many investigators still construct experiments to add support to a current hypothesis or perhaps disprove it. Such experiments may require only one run of the experimental machinery. Any subsequent runs are just to verify the result from the first demonstration.

So, when I say that my research area is experiment design, people ask, “You mean you show people how to be very careful when they set up their experiments?” Certainly careful setup, observation and measurement are important in any kind of experiment. But this is not what experiment design is about.

I want to explain what experiment design is. I also want to motivate the use of designed experiments in engineering problem solving. But first I need to point out that today’s products and the processes for making these products are much more complex than they were even a decade ago. The manufacture of the processing units that go into your cell phone may require dozens of steps, each having multiple controls. Even making toilet paper is a high-tech process!

Engineers are responsible for both defining products and building systems or processes that work. This requires attention to detail and ingenuity. Competitive pressure means tight development schedules and careful cost considerations.

The first step on the road from concept to products in stores involves building and testing prototypes. One way to test a prototype product, system, or process is to change it in some way and measure the effect of that change on some measure of performance. In experiment design parlance, measures of performance are called responses. Factors are aspects of the prototype that we can change or control. Usually, we are interested in the effects of more than one factor, which requires multiple tests of various prototypes. Here, experiment design is about how to decide which prototypes to test and how to go about testing them.

Initially, the goal is to determine which factors have most substantial effect on a response. A brainstorming session may identify a dozen or more factors that we think might affect the response, but generally, only a few factors account for the vast majority of the variation in the response. Imagine trying to differentiate the effects of a dozen factors by trial and error. This is a common practice for engineers who have no experience with experiment design. Trial and error studies often waste resources and give ambiguous results. An expensive approach is to test all possible combinations of the factors. But with a dozen factors, this approach requires 4,096 trials, which is usually prohibitively time consuming and expensive. By contrast, a well-designed experiment can provide clear results while dramatically reducing the required amount of testing.

For example, screening experiments separate the vital few factors from the trivial many.  Experiment designs constructed for factor screening generally require only a small investment  in testing that is between one and two times the number of factors. The ability to distinguish the effects of so many factors with such a small number of test runs is due to the elegant structure of experiment designs for screening.

Until recently, engineers wanting to use experiment designs for factor screening had to rely on catalogues of designs in textbooks or other reference materials. These design tables were restrictive because they were only available for certain combinations of numbers of factors and test runs.

Nowadays, commercial software like JMP can construct screening designs on the fly based on the requirements of each unique study. A computer-aided approach to experiment design provides several advantages over the use of reference designs. Computer generation of designs is immediate. This allows for creation and comparison of multiple design alternatives. An additional benefit is the flexibility of creating a design for each unique problem rather than reformulating your problem to match the requirements of a reference design. Finally, the training or knowledge requirements for appropriate use of computer-aided designs are lower than those for traditional designs. This makes this technology potentially accessible to any engineer.

Imagine a world where the use of experiment design was a standard engineering procedure. Products would get to market faster at lower cost and with higher quality and reliability. Such a world would be a better place to live.

Article Tags