Subscribe Bookmark
ryan_lekivetz

Joined:

Nov 1, 2013

Why it's important to brainstorm factors and levels in a designed experiment

The best time to plan an experiment is after you’ve done it – R.A. Fisher

If you’ve read through my previous blog posts, I usually mention issues discovered during an experiment that I would change if I were to do the experiment again, or things to consider in the subsequent experiments. While I sometimes mention the struggles with choosing an appropriate response, I don’t typically dwell on the importance of choosing the factors and levels. However, choosing ranges poorly and neglecting to consider some factors can diminish the value of the results from an experiment.

three customized diecast cars

I could have gotten more bang for my buck in my designed experiment had I brainstormed the factors and levels with a colleague first. (Photo courtesy of Caroll Co)

A recent experiment I blogged about involved dyeing toy cars based on a number of suggestions I found online. After the first blog entry was posted, there was a knock on my office door from my colleague Lou Valente. He had some ideas for how I might modify the design through different factor ranges and additional factors.

Unfortunately, the experiment was already complete and the data collected, but the experience was a useful reminder of the value in soliciting feedback from different members of a team, particularly when you have access to a domain expert. Lou’s chemical manufacturing experience and passion for DOE gave him insights that escaped my searches. A short discussion with Lou provided far more ideas and understanding of the situation than the few hours I spent searching online.

It can also be useful to have a fresh set of eyes of someone without the expertise, since they may have ideas that wouldn’t even occur to an expert. I won’t delve into the ideas that Lou and I talked about right now, but you can expect to see some of the ideas reflected in a future blog post. I have already been purchasing extra cars for the next experiment.

Fortunately, we did have some positive results in the toy car experiment. Even if the results were not ideal, experimentation is a sequential process. A designed experiment leaves you with more knowledge about the system that you had before, and almost always provides directions to look to for the next experiment.

I was lucky in this case that my valuable reminder wasn’t all that expensive -- the experiment didn’t cost much outside of a few hours of time and some toys cars and fabric dye. But I would have gotten a better bang for my buck if I had talked with Lou first. I’m curious to hear comments as to how you like to choose factors and levels when you experiment.

Who would have guessed that information you gather online isn’t always reliable?

2 Comments
Community Member

David Long wrote:

I would like to see Lou's suggestions for your design. I liked the design; wondered about vinager levels With that many factors and levels - you did well to separate some key main effects - and as you say - another design following the findings of the 1st would be in order - maybe 2 with the second hitting on optimization.

Ryan Lekivetz wrote:

The vinegar was actually one of Lou's ideas - I was only using 1 tablespoon of vinegar, which in hindsight should have been substantially more. Other ideas from Lou include acetone and alcohol, which also could make for some interesting constraints if we consider varying the amount of heat. I like your thinking on the followup experiment(s). Thanks for reading and for the comment!