cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Submit your abstract to the call for content for Discovery Summit Americas by April 23. Selected abstracts will be presented at Discovery Summit, Oct. 21- 24.
Discovery is online this week, April 16 and 18. Join us for these exciting interactive sessions.
Choose Language Hide Translation Bar
ataylor
Level III

Help with Reading a Connecting Letters Report

For pairwise comparisons, I understand the core idea in the Connecting Letter Report, is that if two groups don't share a letter they are significantly different. In the example posted here, I don't understand why we need both of the two first columns --- the second column appears to make the first one redundant since the second column tells us that all pairs between grp_0, grp_1, grp_3, grp_4, grp_5 are not significant and the first column is just the subset of groups 3, 4, and 5, which is covered by the list above.  

 

Also to confirm, is it a correct interpretation to say this report tells us that the only pairing that are significant are grp_2/grp_3 and grp_2/grp_5?

1 ACCEPTED SOLUTION

Accepted Solutions
ataylor
Level III

Re: Help with Reading a Connecting Letters Report

I decided my interpretation was correct that this table shows there is some sort of bug/error in the Games-Howell plugin and I've posted in the comments for that plugin about this issue.

View solution in original post

4 REPLIES 4

Re: Help with Reading a Connecting Letters Report

Most of the time when you see a connecting letters report that seems "nonsensical", the groups typically have different sample sizes and/or the assumption of equal variances has been violated. Are either of these the case for your data? Also, the type of multiple comparison test that you used can play a part on this, too. Which test were you using?

Dan Obermiller
ataylor
Level III

Re: Help with Reading a Connecting Letters Report

This is actually coming from the Games-Howell plugin (so the unequal sample sizes and unequal variance should be handled).  We have some proprietary data where we have evidence the letters report in that plugin has bugs/errors.  I generated these data specifically to illustrate that and wanted to make sure that my understanding of this being wrong is correct before I post about there being errors in the plugin. 

ataylor
Level III

Re: Help with Reading a Connecting Letters Report

And interestingly, if I run Tukey in the main program on the same data, the letters report matches the p-values returned and does not have these oddities, so it is definitely something going on in the Games-Howell plugin. 

ataylor
Level III

Re: Help with Reading a Connecting Letters Report

I decided my interpretation was correct that this table shows there is some sort of bug/error in the Games-Howell plugin and I've posted in the comments for that plugin about this issue.