How do you like your coffee? I like my Coffee Crisp.
Of all the commercials from my childhood, this is one of the few catchphrases that I remember. I suspect most readers have no idea what I’m talking about – (although it does exist in the UK and for a short period of time was available in the US). A Coffee Crisp is a delicious crispy-wafer chocolate bar that is quite common in Canada. If you’ve been reading my blog posts, you probably see where this is going….
After the potato chip smackdown had mixed reviews among our colleagues, mainly due to some not-so-popular flavors, Melinda and I felt like we should make it up to them with one last tasting for the year with something a bit more forgiving to the taste buds.
On a recent trip to Canada, Melinda’s husband was gracious enough to pick up some popular Canadian chocolate bars for a chocolate bar smackdown.This time we had a bit of twist in creating the experiment. We had seven Canadian chocolate bars (including a candy-coated chocolate):
While we could have thrown in some popular US chocolate bars and created a design similar to the potato chip smackdown, this time we wanted each taster to make a direct comparison between each Canadian bar and a similar US chocolate bar (most of the US counterparts are available in Canada as well):
Coffee Crisp / Kit Kat
Crispy Crunch / Heath Bar
Mr. Big / Snickers
Wunderbar / Butterfinger
Caramilk / Rolo
Aero / Hershey’s Air Delight
Smarties / M&M’s
This sounds simple enough – just give each person seven pairs of chocolates and ask them to pick their favorite among each pair. But doing that leaves a lot of information on the table. If we just add one more chocolate bar to taste for each pair, we can ask for the best and worst from each triplet. We still get our head-to-head results, but can also get a general ranking. With limited resources, the standard errors might be large, but the best and the worst may still show up.
How to make this design?
This almost sounds as if we don’t need a design. We could just randomly pick a chocolate bar to go with each pair, but this complicates data collection, and likely won’t get a good spread of different chocolates as the third choice among the individuals.
Based on our previous choice experiments, four surveys seemed like a good number to choose. So we have four surveys with seven choice sets of size 3 in each. Two of the choices in each choice are already determined, since we want each of the US/Canada groups (what I’ll call the primary group) compared. But what about the third choice?
i. For a choice set, all three chocolates should be different. So, the third choice shouldn’t come from the primary group (or else you’re comparing a chocolate bar to itself).
ii. While not necessary, it would be nice if each survey has one item from each primary group show up as a third choice.
iii. The third choice should be a mix of US and Canadian bars.
We addressed the above issues using a Custom Design. We created a seven-level factor called primary,which contains levels of the US/Canada chocolate bars that we want to compare directly. To represent the third choice, a second seven-level called secondary grouphas the same seven levels as primary, and a two-level factor called origin, with levels US and Canada. The combination of secondary choiceand origin determine which bar will be the third choice, and having their effects in the model creation will address points ii and iii above.
To make sure that we’re not giving the same chocolate in a choice set, we only need to make sure that the primarylevel is different than the secondary group. We could go through each combination in the Disallowed Combinations Filter, but a much simpler way is to use a disallowed combinations script that simply says the primary group cannot be the same as the secondary group. That is, we want to disallow primary == secondary group. And that’s all we need to put in the Disallowed Combinations Script:
Now we need to set up the design generation. We need random blocks of size 7 (representing the number of choice sets in a survey), since each row in this case represents a choice set. With four surveys, we need 7*4 = 28 runs, so the design generation looks like this:
Hitting the Make Design button creates a design for us. In this example, we ended up finding a design that had the properties described above.
I thought this was a nice example to show how you can use disallowed combinations in a way you might not always think of. Melinda will share the results next time, including an improvisation that needed to be made when one of the helpers (namely me) was a real butter fingers and dropped all of the Coffee Crisp pieces partway through the experiment. Thanks for reading!