Why one-factor-at-a-time experimentation does not work
Created:
Phil Kay had no idea he'd get such a big response when he started a recent LinkedIn post with the words "OFAT Does Not Work." As the Learning Manager on JMP's Global Enablement team and a data analytics evangelist, Phil often shares nuggets of wisdom with his followers on LinkedIn.
But this post was different, garnering more than 30,000 views, 300+ reactions and 60+ comments. You can read it in full below.
I took the opportunity to ask Phil a few questions – about what's so wrong with one-factor-at-at-a-time experimentation, what's so good about design of experiments, and what he hopes to achieve with his outreach on LinkedIn on the topic.
So what’s so bad about experimenting one factor at a time?
Phil: The problem with experimenting one factor at a time is that nature does not play by those rules. The physical/chemical/biological world is much more complicated. The effect of one factor will depend on how other factors are set. Increasing temperature might have a different effect depending on what pH you are set at, for example. You can’t learn about these important and complex behaviors if you only vary one factor at a time. The end result is that you don’t find the best solutions, and you don’t learn anything much about how to control your process or system to consistently achieve high-quality outcomes. It’s just such a waste of time and effort.
OK, why do people still experiment one factor at a time then?
Phil: Most scientists and engineers have been told from very early on in their education that one factor at a time is the only way to experiment. The idea that you can, and should, vary all the factors at the same time is scary.
All right. Now tell us what's so good about design of experiments, please.
It enables you to find the best solutions by fully exploring the possibilities of your process. And it is not just about seeking the optimum. It is a learning tool: You can capture the complex behaviors and understand the effect of all factors. Using the “magic” of statistics, it also ensures efficiency because you only test the few settings needed to give you the model of the whole system that you require.
What is #DOEbyPhilKay on LinkedIn?
Phil: It’s a hashtag that people can follow to see weekly posts with thoughts and top tips on Design of Experiments (DOE). The posts are a mix of illuminating quotes about DOE, introductions to technical concepts like power analysis, and talking points that occur to me while I’m walking the dog.
Why are you creating these #DOEbyPhilKay posts? What do you hope to accomplish?
Phil: Aside from shameless self-promotion, I hope that the posts provide encouragement and support to people that are just learning to use DOE. I also want to harness the passion of other DOE evangelists, so we can work together towards a vision of every experimenter in the world having the skills to benefit from the method. Maybe more of the educators out there will be inspired to teach DOE to our future scientists and engineers.
What has been the response to your #DOEbyPhilKay postings?
Phil: It has been great to find out that I am not the only one that feels strongly about this topic. The OFAT post has had over 30,000 views! I’ve really enjoyed the discussions. And it has connected me with loads of new people in different industries around the world. Someone recently told me that they teach a module on DOE for chemistry undergraduates, and they begin the course with a quote from one of my articles – probably one of the proudest moments of my career.
You can follow Phil's posts on LinkedIn by clicking here and tapping the "Follow" button. (Note: You must be signed in to LinkedIn first.) And now please enjoy this picture Phil took of his adorable dog on one of their walks in the gorgeous English countryside.
Comments