Improving Equivalence Tests in Fit Y by X

[jszarka]

I've been a big fan of equivalence tests in JMP. For a one-sample scenario in the Distribution platform I like the output because it has a nice graphical representation of the equivalence bounds overlaid with the confidence intervals on the mean. You also get a deliberate interpretation statement at the bottom.

In a multi-sample scenario in the Fit Y by X platform it's less interpretable. I was in a meeting with a few engineers, and they had no idea how to interpret the two normal distributions and indicated they definitely prefer the one-sample case.

My proposal would be to create similar output in Fit Y by X as to what you see in the Distribution platform. For the example above, the equivalence bounds of [-6, +6] can be easily shown with a red confidence interval based on the actual output (centered around 3.02).

[Jeff_Perkinson]

In JMP 16 we are improving the equivalence test in Fit Y by X as suggested here.

[bayesfactor]

I have a somewhat related request. We often use a contingency plot to show proportional outcomes between groups. An example yield plot for 3 groups is shown below (yield is "good"=G or "bad"=B). It would be super useful to include confidence intervals on this plot so that users don't have a false sense of difference in groups from looking at this raw plot. Looking at this plot doesn't give a sense for whether the groups are different or not different, depending on the N per group.

[Jeff_Perkinson]

The Equivalence Tests report in the Oneway platform is much enhanced in JMP 16:

 

[PatrickGiuliano]

Here's some wonderful literature that helps explain what this new "diffogram" Plot is and how to properly interpret it, courtesy of none other than jmp customer support! 

"It [the diffogram] is most frequently used in pairwise comparisons of means, but can also be used to visualize the results from equivalence testing.  See the JMP documentation of the diffogram for multiple comparisons here: https://www.jmp.com/support/help/en/16.0/#page/jmp/multiple-comparisons.shtml?os=mac&source=applicat.... 

There is a more detailed explanation of the plot on pages 3-5 of the SAS paper here: https://support.sas.com/resources/papers/proceedings14/1902-2014.pdf. This paper also demonstrates the plot that you suggested (one-dimensional confidence intervals on a number line with the equivalence region).  The diffogram  is described in more detail in this book:  Hsu, Jason (1996) Multiple Comparisons Theory and Methods, Chapman & Hall/CRC: Boca Raton. But you might not have free access to this.

In the diffogram, the X-axis plots the mean for the first group and the Y-axis the mean for the second group.  There is also an implied third axis on the diagonal which shows the difference between the means and a confidence interval for the difference. The key advantage that the diffogram has over the suggested one dimensional plot, is that it plots the means of the two groups being compared (instead of only showing the difference) and this provides a natural ordering for the confidence intervals.  The utility becomes most apparent when many comparisons are performed.  The diffogram concisely plots many comparisons.  Note that you can also select a point and see the corresponding row in the Equivalence TOST tests table highlighted."