In my previous blog post, I introduced the paper helicopter experiment. You saw how it is a great way for anyone to get some hands-on learning about the value of the design of experiments (DoE). I also mentioned that we set a competition based on this exercise and asked for experimenters to submit a report on their learning for a chance to win a DJI Mavic 2 drone. Now, let's find out about our competition winners and hear from them about what they learned in the experiment.
The winners are Bruce and Amelia Alexander. The judges thought they really captured the spirit of the competition. Their write-up, Everything is Awesome: Guinea Pig Health and Safety, Design of Experiments and Crashed Copters, demonstrated effort, thoughtfulness and a great sense of fun. I will let them speak for themselves with some excerpts from their winning report...
Tell us about yourself.
Bruce Alexander is a lecturer in physical chemistry at the University of Greenwich. Originally trained as a spectroscopist, has interests in solar hydrogen production, instrumental analysis, formulation science, and has recently been making a range of cosmetics with his MSc Formulation Science students. He has a passing acquaintance with Design of Experiments (although if his students read this, he is an expert). Amelia Alexander is about to finish her third year at school and is looking forward to the summer holidays! Her guinea pigs, Blaise and Hamish, are looking forward to reduced helicopter flights in the near future.
What did you learn?
taller child to launch the helicopter longer flight path.
Spawn-the-elder getting into design of experiments with the #JMPhelicopter challenge. Spawn-the-younger, not so much. pic.twitter.com/zroE4voaPc
— BD_Alexander (@bd_alexander) July 12, 2019
How did you make sure that you got useful data? Describe what you did so that another experimenter could repeat your results.
The launch pad was created by one of the team members standing on a dining chair (Ikea, Lakeside). The team member was 135 cm tall and the launch height of over two meters was achieved by the team member standing straight with her arm upstretched. Health and Safety implications were considered: experiments were run when the health and safety manager (a.k.a. mum) was at work, resident guinea pigs were protected by a suitable enclosure.
Helicopters were held in the tips of two fingers by the team member and dropped after a countdown to launch. At this point the clock was started. Timings were recorded for when the helicopter crash landed assuming that it had landed without bumping into the launch pad, team member, or the guinea pigs’ cage. Flight times were recorded in triplicate. Any helicopters that had been nibbled by the resident guinea pigs were scrapped and replaced (n = 1).
Factors investigated were: construction material of the helicopter, wing length, wing width, and wing geometry. The last factor considered wings that were at 90° to the helicopter body (termed “flat”) and wings that were folded at an angle of 30° to the helicopter body (termed “twisted”). This was a necessary variable after some of the paper helicopter wings bent upon crash landing and extraction from the guinea pig’s cage. No cavies were harmed in this experiment.
We did try to add pilots that could fly to the #jmphelicopter challenge. Batman was not impressed. Nor could he fly very well. pic.twitter.com/apQcIpxHTS
— BD_Alexander (@bd_alexander) 12 July 2019
How could you improve on the experiment?
A taller experimental team would be useful for longer flight times. This might allow us to increase the signal to noise in some of the experiments. Ideally, a larger drop zone could be used, but neither of the team fancied continually running up and down stairs to retrieve crashed helicopters although this would have been an alternative to the “two levels” employed herein.
I think that these improvement ideas are really important. In fact, the experimenters at the customer visit that I mentioned in my previous post came to the same conclusion. To get the most insight from this experiment, you should maximise the drop height to get the best signal to noise. It is testament to the DoE method that Bruce and Amelia still got useful learning, even with some quite variable recorded results. Without DoE, I suspect that this is a case where you could easily end up "chasing noise."
Bruce and Amelia included a table of results in their PDF report (wouldn't it be great if you could open this table in JMP? Watch out for new features coming in JMP 15 in October). I have used their data to recreate some analyses as interactive visuals in JMP Public. You can find the full model report with interactive Profiler and also a variability chart suggesting that repeat flights may have damages the helicopters. You can also download the data.
Another interactive visual is the following bubble plot. Click play to see the important effect that paper type has on flight time:
I hope that this has inspired you to try the paper helicopter experiment yourself as a way for you and your friends, family, students and colleagues to learn about DoE.