For one of our line of products, we often do taste testing in order to prove that less of it leaches out into the test substance (water) than other products or formulations. The test methodology is to do an extraction into water and then do comparative taste testing by at least 10 people. Based on work that our corporate operations research department did over a decade ago, they estimated that better than 8/10 preference represents a significant difference at p=0.05.
Formulation A vs Formulation B = 11 vs 2 preference
Formulation A vs Formulation C = 8 vs 6 preference
Formulation B vs Formulation C = 5 vs 10 preference
(note that the number of comparison is not the same in all examples, this is on purpose)
Conclusions: Formulation A is better than formulation B; Formulations B and C have no significant difference; Formulation A and C have no significant difference.
My 2 questions:
-Have insights in taste testing improved since then, and
-What would be a good way to setup a test like this in JMP, especially when there are not 3 but 10 formulations and a group of 25 people out of whom a random 15 might do any individual comparison?
We did something along these lines in the JMP group, comparing potato chips. To get more information, we used multiple groups of 3 potato chips, where individuals would specify their favorite and least favorite in each group. Related to what's in JMP 12: the design blog and the analysis blog.