It’s World Statistics Day! To honor the theme of the day, the JMP User Community is having conversations about the importance of trust in statistics and data. And we want to hear from you! Tell us the steps you take to ensure that your data is trustworthy.
Choose Language Hide Translation Bar
Chocolate Bar Smackdown: US vs. Canada

How do you like your coffee? I like my Coffee Crisp.

Of all the commercials from my childhood, this is one of the few catchphrases that I remember. I suspect most readers have no idea what I’m talking about – (although it does exist in the UK and for a short period of time was available in the US). A Coffee Crisp is a delicious crispy-wafer chocolate bar that is quite common in Canada. If you’ve been reading my blog posts, you probably see where this is going….

After the potato chip smackdown had mixed reviews among our colleagues, mainly due to some not-so-popular flavors, Melinda and I felt like we should make it up to them with one last tasting for the year with something a bit more forgiving to the taste buds.

On a recent trip to Canada, Melinda’s husband was gracious enough to pick up some popular Canadian chocolate bars for a chocolate bar smackdown.This time we had a bit of twist in creating the experiment. We had seven Canadian chocolate bars (including a candy-coated chocolate):

  • Coffee Crisp
  • Crispy Crunch
  • Mr. Big
  • Wunderbar
  • Caramilk
  • Aero
  • Smarties (not Rockets, but Smarties)
  • While we could have thrown in some popular US chocolate bars and created a design similar to the potato chip smackdown, this time we wanted each taster to make a direct comparison between each Canadian bar and a similar US chocolate bar (most of the US counterparts are available in Canada as well):

    • Coffee Crisp / Kit Kat
    • Crispy Crunch / Heath Bar
    • Mr. Big / Snickers
    • Wunderbar / Butterfinger
    • Caramilk / Rolo
    • Aero / Hershey’s Air Delight
    • Smarties / M&M’s
    • This sounds simple enough – just give each person seven pairs of chocolates and ask them to pick their favorite among each pair. But doing that leaves a lot of information on the table. If we just add one more chocolate bar to taste for each pair, we can ask for the best and worst from each triplet. We still get our head-to-head results, but can also get a general ranking. With limited resources, the standard errors might be large, but the best and the worst may still show up.

      How to make this design?

      This almost sounds as if we don’t need a design. We could just randomly pick a chocolate bar to go with each pair, but this complicates data collection, and likely won’t get a good spread of different chocolates as the third choice among the individuals.

      Based on our previous choice experiments, four surveys seemed like a good number to choose. So we have four surveys with seven choice sets of size 3 in each. Two of the choices in each choice are already determined, since we want each of the US/Canada groups (what I’ll call the primary group) compared. But what about the third choice?

      i. For a choice set, all three chocolates should be different. So, the third choice shouldn’t come from the primary group (or else you’re comparing a chocolate bar to itself).

      ii. While not necessary, it would be nice if each survey has one item from each primary group show up as a third choice.

      iii. The third choice should be a mix of US and Canadian bars.

      We addressed the above issues using a Custom Design. We created a seven-level factor called primary, which contains levels of the US/Canada chocolate bars that we want to compare directly. To represent the third choice, a second seven-level called secondary group has the same seven levels as primary, and a two-level factor called origin, with levels US and Canada. The combination of secondary choice and origin determine which bar will be the third choice, and having their effects in the model creation will address points ii and iii above.

      To make sure that we’re not giving the same chocolate in a choice set, we only need to make sure that the primary level is different than the secondary group. We could go through each combination in the Disallowed Combinations Filter, but a much simpler way is to use a disallowed combinations script that simply says the primary group cannot be the same as the secondary group. That is, we want to disallow primary == secondary group. And that’s all we need to put in the Disallowed Combinations Script:


      Now we need to set up the design generation. We need random blocks of size 7 (representing the number of choice sets in a survey), since each row in this case represents a choice set. With four surveys, we need 7*4 = 28 runs, so the design generation looks like this:


      Hitting the Make Design button creates a design for us. In this example, we ended up finding a design that had the properties described above.

      Final Thoughts

      I thought this was a nice example to show how you can use disallowed combinations in a way you might not always think of. Melinda will share the results next time, including an improvisation that needed to be made when one of the helpers (namely me) was a real butter fingers and dropped all of the Coffee Crisp pieces partway through the experiment. Thanks for reading!

      Article Labels

        There are no labels assigned to this post.


      Shannon Conners wrote:

      As both a participant in the experiment and a big fan of chocolate, I'm interested to see part 2! I am going to make a wild guess that the not-really-Smarties didn't go over so well. I am thinking maybe a comparison of dark chocolates with add-ins (e.g., chili, mint, caramel, almonds, sea salt, etc) might be a good next experiment. Just saying.


      Ryan Lekivetz wrote:

      You might be right about the Smarties - I guess you'll find out soon ;)

      I think your chocolate idea is a great one. I actually have some chocolate in the pantry to test out making truffles with different ingredients. It's just a matter of finding the time.

      Thanks for reading!


      Wayne Levin wrote:

      Great analysis Ryan except for one, gaping omission: Oh Henry!!! There's nothing like it. It trumps all other choices - especially the peanut butter version. It apparently originated in the US but I haven't been able to find it there for years. Of course, it's very popular in Canada - made by Hershey.


      Ryan Lekivetz wrote:

      You're absolutely right, Oh Henry would have been great to have in there! I haven't had the pleasure of trying the peanut butter version - I'll keep an eye out next I'm back in Canada.

      Thanks for the comment!