DOE to test taster's ability to predict color of candy (for kids)

[#1]

Dear JMP & Community,

  During this time of being stuck at home with kids and working on school projects, I thought of running a "taste test" with my son who recently claimed that he could tell the color of a chocolate candy based on the color of the candy shell (it's a candy similar to that of M&M).

  I'm not entirely sure how to set up the DOE because it doesn't quite fall under Choice or MaxDiff Designs, or the Covering Array or Nonlinear DOE. It's not quite the same as the potato chip smackdown blog or the chocolate smackdown blogs here and here.

  I simply want to test whether or not my son can truly (in a blind and randomized trial of the candies) taste the flavor of the coloring and predict it correctly.

  At present, I've only come to the following idea of using the Custom DOE with one X1 factor: Actual Color, and the response being Predicted Color: 1 if predicted correctly, 0 otherwise. Then, by looking at the distribution of the response column, and looking at the Proportion Zero and Proportion Nonzero, we could get an idea of how often he was right vs. wrong.

  That's pretty easy, and straightforward, but I think it might be interesting to instead somehow record the response and analyze the DOE in such a way that we could see if he actually is able to taste the difference in certain colors, say always getting the Blue and Purple correct, maybe sometimes with Green, but never with Purple.

  At first, this whole project sounded easy, but the more I think about how to design and analyze it for learning more valuable information (than just whether he's right or wrong), the more I see it's complicated, at least to me. It's not so straightforward to me on how to set it up to get some really interesting information.

  If anyone has some suggestions, I appreciate your feedback!

Thanks,

DS

[#2]

I will just say that no amount of empirical evidence will convince any kids that their theory is wrong!

[#3]

OK, seriously, your idea for the design seems perfect. I would roughly take the number of levels and multiply by 5 to get the number of runs. (He won't mind eating more candy, eh?)

Change the response column to be Nominal modeling type. You can delete the Response Limits column property, too. This analysis is best handled in Analyze > Fit Y by X. That direction will take you to the Contingency platform.

[#4]

Hi @Mark_Bailey,

  You're absolutely right about evidence not changing his mind! I thought it might be a fun exercise for him to get some experience with DOE and then understanding the outcomes of the analysis. And, he is more than fine with eating more!

  I also thought about the change in the response column, and deleting the response limit to analyze in the contingency platform. I might actually do both so as to see the general distribution as well as the contingency.

  Regarding the design, do you think I should block it for him to "take a break" so as not to influence the decision making process by tasting too much of the same chocolate? Maybe block it in groups of 5 or so -- four blocks?

  Thanks for your thoughts!

DS

[#5]

Blocking is not a bad idea. Use random effect blocks in this case.

Another idea: first demonstrate the null hypothesis. My daughter came home with an assignment in the second grade. They were given a problem (all information provided) and told to perform a t-test to decide about the difference in the height of plants under two treatments. I was ecstatic! At first. Then I realized that there was no understanding or learning, so I used the Socratic method to take her through the mental steps of the hypothesis test. She nailed it.

So I suggest setting something up where the response is random. Make it clear that the assignment of color is purely random with no association with the true color. Do this part more than once. Then have him classify the color of the candy. Show him the results and remind him about how the assignment might have been random. In fact, the basis for the test (i.e., the null) is that it is random.

I'm anxious to hear how your explanation of likelihood ratio tests goes over...

[#6]

Hi @Mark_Bailey, @P_Bartell, @statman,

  I'm happy to report that we have conducted the experiment, and it was a lot of fun! I printed out the runs and my wife had a great time conducting the experiment for us -- she managed the candies, water, coffee grounds, and recording our responses. We both stayed blindfolded the whole time, although could hear each other's responses. I'm attaching the data table, but also just commenting on some of the initial results. As a reminder, there were three factors: the actual candy color (blue, green, pink, purple), tester (A and B), and cleaning method (water swish or smelling coffee grounds)

Overall:

1. the DOE selected each color and cleaning method equally, however shifted one run to one tester over the other, so the testers were not equally chosen.
2. As a collective, the two testers were correct only 28% of the time, not much different than the 25% random chance (no surprise there).
3. Pink was the least frequently predicted color. In fact in the overall heatmap, it's seen that pink was also actually NEVER correctly chosen. If there is a flavor difference, it was one that both testers mixed up with other colors.
4. Contingency analysis results in a likelihood ratio of only 8.4, so pretty small (again, not surprising).
5. Both testers had an equal number of correct answers (4), although unequal number of runs.
6. Overall heatmap shows that blue was correctly chosen the most of all the colors (4 times) with green and purple being equally chosen correctly (2 times each).

By Tester:

1. Supposedly, tester A was correct 31% of the time compared to tester B with 27%, but I think this is misrepresents tester A's ability to correctly identify the color because tester A had only 13 candies total and not 14 like tester B.
2. Both testers had nearly identical distributions in their frequencies of color choices. So, perhaps this is an indication there could be some kind of flavor difference, but not one that either tester is trained to detect.
3. Tester B had better results refreshing their senses using coffee grounds compared to water 75%/25%, whereas tester A was 50/50 between the two.
4. A heatmap by tester suggests that tester B could differentiate between more colors (maybe), as that tester was able to correctly determine 3 of 4 colors, although accuracy was still low. Tester A could only correctly distinguish two colors, but also with low accuracy.
5. A contingency by tester analysis again shows very small likelihood ratios, but indicates that tester A is twice the ratio. I'm not sure about this because of the uneven number of runs between the testers, but maybe the analysis takes this into account?

Any other thoughts would be much appreciated. By the way, I used the JMP 16EA1 to analyze the results (I'm running calculations with JMP Pro 15.0 and don't want to interrupt them).

  I think the best thing that came from this is that my son is now excited about doing a DOE -- like a taste test for preference of ice cream toppings or other candies with his friends. So hey, even though we didn't really find all that much from this DOE, it got him interested in doing more and trying something new. For a 10 year old, I think it's great!

Thank you all for your input!,

DS

[#7]

@SDF1Very cool. I don't know why I didn't think of this before but this problem is very similar to the famous "Lady Tasting Tea" narrative from none other than R. A. Fisher.

https://en.wikipedia.org/wiki/Lady_tasting_tea

Years ago when my kids were smaller I was an instructor in all of Eastman Kodak Company's Six Sigma educational programs. We had all manner of cool toys to play with. I would often trial our training modules with a new toy with our kids. We figured if a 8 year old could get it...an engineer at Kodak has a fighting chance too. I'd also take them to their grammar school for demonstrations with the kids. Two that were most fun for the kids were the 'Statapult' and paper helicopters. The statapults are a bit pricey...but the paper helicopters are dirt cheap. Just need paper, some scissors and maybe a few props like paper clips and a stopwatch. The kids had a ball and loved when Mr. Bartell came to play with us...and they learned a thing or two about statistics as well!

Here's a link for the helicopters:

https://williamghunter.net/george-box-articles/teaching-engineers-experimental-design-with-a-paper-h...

And the statapults:

https://sixsigmaproductsgroup.com/product/the-statapult-catapult/

[#8]

Hi @P_Bartell,

  Thanks for the links, those are very interesting reads. The statapults are a bit pricey, but look very cool. From the manual, the design seems pretty simple, and one that I might be able to put together with him -- might be a good Scout project for him and I to work on. In the meantime, I think the flying helicopter will be a great example, and he'll have a fun time with it.

Thanks again, everyone, for the feedback in information!

DS

[#9]

I'm a bit confused by this in your post:

" I thought of running a "taste test" with my son who recently claimed that he could tell the color of a chocolate candy based on the color of the candy shell"

I assume you mean he can guess the color based on the taste?

First what Mark said.  I will add there is difficulty in assessing measurement errors, especially when it comes to sensory perception.  You could add another tester (another sibling or you to the study).  You could also think about how are the taste buds "cleansed" between samples? This might allow you to add another factor to the design.  How about bag-to-bag variation of the candy itself?  Does it vary?  What about other brands?  Great idea to teach factorial designs to kids before they get slammed with OFATs in academia.

[#10]

Hi @statman,

  Yes, sorry about the typo. His claim was that by tasting the candy, he could tell the color of the candy shell.

  Right now, we have only a single bag of the candy. I do like the idea of adding a second tester to the mix and maybe even cleansing method.

  I think this will be fun, especially for him!

Thanks for the feedback.

DS