Why it's important to brainstorm factors and levels in a designed experiment
The best time to plan an experiment is after you’ve done it – R.A. Fisher
If you’ve read through my previous blog posts, I usually mention issues discovered during an experiment that I would change if I were to do the experiment again, or things to consider in the subsequent experiments. While I sometimes mention the struggles with choosing an appropriate response, I don’t typically dwell on the importance of choosing the factors and levels. However, choosing ranges poorly and neglecting to consider some factors can diminish the value of the results from an experiment.
I could have gotten more bang for my buck in my designed experiment had I brainstormed the factors and levels with a colleague first. (Photo courtesy of Caroll Co)
A recent experiment I blogged about involved dyeing toy cars based on a number of suggestions I found online. After the first blog entry was posted, there was a knock on my office door from my colleague Lou Valente. He had some ideas for how I might modify the design through different factor ranges and additional factors.
Unfortunately, the experiment was already complete and the data collected, but the experience was a useful reminder of the value in soliciting feedback from different members of a team, particularly when you have access to a domain expert. Lou’s chemical manufacturing experience and passion for DOE gave him insights that escaped my searches. A short discussion with Lou provided far more ideas and understanding of the situation than the few hours I spent searching online.
It can also be useful to have a fresh set of eyes of someone without the expertise, since they may have ideas that wouldn’t even occur to an expert. I won’t delve into the ideas that Lou and I talked about right now, but you can expect to see some of the ideas reflected in a future blog post. I have already been purchasing extra cars for the next experiment.
Fortunately, we did have some positive results in the toy car experiment. Even if the results were not ideal, experimentation is a sequential process. A designed experiment leaves you with more knowledge about the system that you had before, and almost always provides directions to look to for the next experiment.
I was lucky in this case that my valuable reminder wasn’t all that expensive -- the experiment didn’t cost much outside of a few hours of time and some toys cars and fabric dye. But I would have gotten a better bang for my buck if I had talked with Lou first. I’m curious to hear comments as to how you like to choose factors and levels when you experiment.
Who would have guessed that information you gather online isn’t always reliable?