Subscribe Bookmark


Nov 1, 2013

Chocolate smackdown: US vs. Belgium

Plates with chocolate from Belgium and the US

Preparing for our chocolate smackdown

This year, I was fortunate enough to present at the JMP Discovery Summit Europe in Brussels, Belgium. When looking for something to bring back for my colleagues in Cary, I first thought of chocolate (actually, my first thought was probably beer, but that didn’t seem like a good idea). One of my colleagues on the trip, Melinda Thielbar, had blogged about the probability and choice profilers in JMP 12. So we thought instead of simply giving our colleagues chocolates, why not make them work for it and collect some data at the same time? As an added bonus, we might just get to blog about it.

Before leaving, we picked up three types of plain chocolate bars from a Belgian chocolatier having 41 percent, 60 percent, and 65 percent cacao content. Upon our return to the US, Melinda found chocolate from high-end US chocolatiers with the same cacao content. Would members of the JMP team show a preference for US or Belgian chocolate? Milk or dark?

We had the basic premise of the experiment: a two-level factor for the chocolate origin (US/Belgium) and a three-level factor for the cacao content (40/60/65). If I go into DOE-> Choice Design, I can add these two factors and their interaction in the model controls. But we still had to figure out the rest of the choice design setup under the Design Generation tab. Our resources were also constrained. We had six different chocolates, with roughly 40 pieces of each at our disposal. Our design generation tab looked like this:


How did we decide on those numbers?

  • Number of attributes that can change within a choice set: Two. Both factors could change within a choice set.
  • Number of profiles per choice set: Two. Give tasters two pieces at a time and let them pick the one they liked better.
  • Number of choice sets per survey: Five. We didn’t want to overwhelm people with too much chocolate. Plus, we had a limited number of pieces. Five comparisons with 10 total pieces of chocolate was our ultimate choice.
  • Now, how many surveys?

    Ten pieces of chocolate per survey and around 240 pieces of chocolate meant that we could recruit 24 people to taste. With six different chocolates, there are 15 (6 Choose 2) different pairwise options that could occur. This suggested that three surveys was a good choice for getting a variety of pairwise comparisons among our tasters. So:

    • Number of Surveys: Three.
    • Expected number of respondents per survey: Eight. With 24 respondents total, we get 24/3 = 8.
    • If we click Make Design, we can start setting up the tasting. But...

      Another design approach using the custom designer

      Using the choice designer with the setup just described, we end up with a design that has each of the six chocolate types (2 origins x 3 cacao contents) equally balanced when viewed over all three surveys. This is what we wanted, since we had roughly the same amount of each type of chocolate. However, if you look at individual surveys (i.e., what each taster would be getting), you might see a disproportionate selection of origin or cacao content. We wanted a design that maintained the overall balance, but also had a nicer balance for each individual taster.

      Since this notion of balancing out the factors is important in our case, we can use the custom designer (DOE-> Custom Design) to achieve our goal. We do this by treating the survey and choice set as nested random blocks, similar to what Bradley Jones did in a previous blog post. We use a false VHTC factor for the surveys and a false HTC for the choice sets. Our factors table looks like this:


      As that blog post by Bradley shows, we want to remove the effects for false VHTC and false HTC. The purpose is to have the surveys and choice sets treated as random effects, so we can remove the fixed effects part (hence the “false” piece of the name). The model dialog looks like:


      For the Design Generation dialog, we’re just going to create the basic setup for the three surveys, which would be replicated eight times afterward. With three surveys, we have three whole plots, and five choices sets per survey means 15 subplots. Each choice set has two profiles, so we need 30 (2 x 15) runs:


      The resulting design is just what we wanted! It has equal use of each of the six chocolates, and an even split of US and Belgian chocolate for each survey with the cacao content being close to balanced.

      Keep in mind that the model used to create this design was not a choice model. Using this method does require setting up the Choice platform manually from Analyze -> Consumer Research -> Choice, which will be discussed in a future blog post.

      Final Thoughts

      While I would still recommend making designs for choice experiments in the choice designer, I thought this was a nice example of the flexibility of the custom designer. The design creation using the custom designer was the one used to conduct the experiment.

      You will hear about the experiment results from Melinda in a future blog post. I can reveal that we did learn JMP team members like chocolate – we found 24 volunteers in less than 30 minutes after the call went out!

      Now it’s time for me to start writing an abstract for the next JMP Discovery Summit Europe in Amsterdam -- hopefully, I’ll see you there!

      Community Member

      Michael Clayton wrote:

      Age of judges would be key in my opinion.

      My personal studies over many decades has shown me that your 40/60/65 values would correlate with age of judges. As I approach 80, I only buy 78-82 % cacao dark chocolate (and I prefer the darker malt beverages.). My global travels as a migrant worker has allowed me to interview people of all ages and nationalities and I have found that as people age, their choice of malt beverage and chocolate moves toward the darker malts and high cacao percentages, regardless of country of origin. I suggest much larger number of judges, and wider range of factors until this age-related mystery is illuminated.

      Ryan Lekivetz wrote:

      Thank you for the comment - I'm quite intrigued by your idea. We did collect some respondent information, but age was not one of the factors. Based on how popular this was with the JMP group, I'm hopeful there will be another chocolate related experiment in the future, and we can certainly consider age (and have more volunteers).

      Community Member

      Pablo Moreno Pelaez wrote:

      Love the experiment!

      Would Belgians prefer American chocolates? Let me know if you want to expand the experiment, I will be glad to help :)

      Ryan Lekivetz wrote:

      Glad that you liked it! Thanks for the offer to help - I may just take you up on that one day =) I think it would be nice to do something at a Discovery Summit... maybe one of these days.