Subscribe Bookmark


May 27, 2014

Does JMP have the Triangle Test?

I started my professional life as an economist. Sometime after I started working with statisticians, I mentioned the “Heckman Two-Step Model” to a colleague. He’d never heard of it, and after a few minutes of talking past each other, I wrote down an equation. “Oh,” he said. “You mean an instrumental variables model.”


So, I’m sympathetic when someone asks “Does JMP have a [Name of Famous Person in Their Field] [Statistical-Sounding Name] [Optional Technical Term from the Field]?” and everyone around me gives them blank stares. Every technical field has its own language, and as Randall Munroe discussed in his keynote at our 2016 Discovery Summit conference in Cary, learning the jargon is part of showing you’re technically proficient in your field. If you mostly work inside your field, you forget that there’s nothing magical about those words and phrases. They’re shorthand for a series of steps or a body of research.


JMP is an all-purpose statistical tool, so if you’re talking about a statistical test, chances are that JMP can do it. It just may not have the name that you’re used to seeing.

The “Triangle Test” from a Statistician’s Perspective

The Triangle Test, for example, is a sensory analysis test where you give a panelist three samples to taste. Two of them are the same formula. One of them is a different formula. The panelist’s task is to tell you which one is different. If the panelist can’t tell the difference (either because the formulas taste the same, or because the panelist isn’t sensitive to the differences), the probability of picking the correct answer at random is 1/3 (one out of three). There are other such tests:

  • The Duo-Trio test is similar, except instead of presenting three samples and asking which one is different, the panelist is given a reference sample and then two unidentified samples. The panelist’s task is to determine which of the two unmarked samples is the same as the reference sample. If the panelist can’t tell the difference, the probability of him or her choosing the correct answer randomly is 1/2 (one out of 2).
  • The Two Out of Five test is exactly the same as the Triangle Test, except the panelist is given five samples. Three are the same sample, and two are different. The chances of the panelist picking both of the different ones correctly is 1/5*1/4. (One out of five, and then given that one of them has already been chosen, one out of four).
  • In the Paired Comparison test, the panelist is shown two samples and asked to choose one. Sometimes the panelist is asked to pick which one tastes better. Sometimes the panelist is asked to pick which one has more of an attribute, for example, the “sweeter” flavor or the “more intense” flavor. If the panelist can’t tell the difference, the probability of a correct answer is 1/2 (one out of two).

All of these tests have different procedures and different purposes in sensory analysis, but to a statistician, they are all chi-squared tests of proportions with a given null hypothesis for each level. The null hypothesis for each of the above is “No Difference.” What changes is the probability of a “Correct” answer when the null hypothesis is true.


For the Duo-Trio and the Paired Comparison, the probability of a correct answer under the null hypothesis is 1/2. For the Triangle Test, the probability of a correct answer under the null hypothesis is 1/3, and for the Two Out of Five test, the probability of a correct answer is 1/20 (1/5*1/4).


JMP does have the ability to do a test like this through the Distribution platform. Below, I’ll show the steps and illustrate an add-in I wrote to make it easy to run the tests in JMP. 

An Add-In for the 4 Common Types of Simple Sensory Tests

The beauty of JMP is that it makes statistics accessible for people who don’t want to think about all of the details of every analysis. In the last few versions, we’ve also made the software easier to customize with add-ins and the add-in builder. I wrote my own add-in to make it easy for people who don’t want to keep a list of the correct mouse clicks and the right proportions for each of the above tests. All you need is a column that says whether the panelist got the answer “right,” and to know which kind of test you want.


You can download my sensory test add-in from the Add-Ins library.


If you’ve never installed a JMP add-in before, don't worry! Our documentation has step-by-step instructions.




When you run the add-in, it will open a dialog asking what kind of test you want and which column in the data table has whether the panelist got the test right. The script has some intelligence in deciding which level is the correct level. If you want to make sure it chooses correctly, see the section titled “How to Set Up the Data.”




How to Do All Four of These Tests in JMP: Details

The example table that I’ve attached contains a simulated Triangle Test. Running the Distribution script shows the proportion of correct answers (“Yes” vs “No”) in the correct answer column and a hypothesis test as to whether there is a true difference.




As you can see, the Hypothesized Prob (the probability under the null hypothesis) of a “Yes” is 0.333333=1/3 and the Hypothesized Prob of a “No” is 0.666667=2/3. 


While the test isn’t on by default, it’s easy to request it from the red triangle menu connected to the distribution of interest (not the main platform’s red triangle menu, but the one below it).


Test Porportions.png


Once you request the tests, JMP will give you a window that lets you enter the probabilities under the null hypothesis. 




How to Set Up the Data

You need a column in your data table that tells whether the person got the test “right.” I usually code the correct column as 0 and 1 (0=no/incorrect, 1=yes/correct). Some software programs code the correct answer as 1 and 2 (1=no/incorrect, and 2=yes/correct). Or, it might be a text field that has “Yes” and “No” or “Correct” and “Incorrect.” Any of these will work, as long as you make sure that the correct proportion is assigned to each value in the hypothesis test.


Tests of Proportions are available for categorical columns only, so make sure the modeling type is set to Nominal in the data table. If you have a numeric code, it’s helpful to add a Value Label to help your user (and you!) remember which level means what. As an added bonus, the add-in I wrote checks the value labels for “Yes” and “No” or “Correct” and “Incorrect” when it decides which data level is associated with correct and incorrect answers. If you’re handing the data table and the script to a user, adding the value labels will make sure they always have the right values.


So yes! JMP has a Triangle Test, and a Duo-Trio Test, and a Two-Out-Of-Five Test, and a lot of other useful tools for analyzing your sensory data.