How do I detect by-location bias to any of my categorical responses ?

[#1]

I use JMP Pro 16.

I have a table with 16k+ people from 7 locations answering 90 Agreement (5 point response scale ) questions.  I have made T2B, B2B, Neutral supercategories of the responses.

The purpose of my analysis at the moment is to detect any bias (by location) to any of the questions.  Knowing this would help in future explanations.. I created contingency tables by location, for each of the 90 responses at 3 levels.  The homogeneity test gave p-values lower than alpha 0.05, indicating to me that at least one of the places does not have the same response frequency as the others.  

Q1: Is there a better way (graphically perhaps) to identify the place or places that are different from the rest?  Other than trying to analyze the letters, that is.

Q2:  Is there a better (proper and hopefully easier) way to achieve my objective?  For perspective, we are dealing with consumer survey, so I'm just trying to tease out the most important signal from an otherwise noisy dataset.

Appreciate the community help in advance.

[#2 · Accepted solution]

I find Multiple Correspondence Analysis to be very helpful in cases with many categorical levels. Here is a simple example using your data to illustrate the basic approach:

[#3]

Hadn't thought of that at all, Mark.  Thanks;  I'll give that a try. 

[#4]

Mark,

I think this would work for me.  Would you please check my thought process?

I could start showing the MCA plot to illustrate the importance to question f2f2122 of B2B response and location r.

I reinforce the statement by graphically showing the partial inertia contribution of B2B and location r to Dimension 1&2.

I might be asked for a threshold statistic (like a Cochran's Q?) to determine next action, but I can say we can align on which questions to potentially throw out after looking at results of 90 questions.

Does this make sense?

[#5]

Did you read the documentation? You use the Y, Response analysis role to create the biplot, which is the main feature of the analysis. Still, you have other analysis roles, such as the X, Factor role or the Z, Supplementary Variable role. The ratings seem like a natural choice for the Y role. The location might make sense in either the Y or X roles.

I admit that I am not clear yet on the nature of all the variables, so I cannot tell if what you propose makes sense,

[#6]

Sorry; I think I confused you.

I've used MCA before and I know how to assign roles.  My last question has to do with my customers asking what to do with the results;  a hard-fast statistic like a threshold value (say, UCL in Hotelling's T2 in outlier analysis) would make them happy.  In MCA, whilst I have inertia and partial contributions, there's no such threshold I can pinpoint.  But that's ok.  I was looking for something more along the lines of qualitative analysis, and I got it.  

Thanks to your suggestion, I now have another use application for MCA!

[#7]

I believe that MCA is a grossly under-utilized technique., so I am always glad to see another application for it! I hope it helps you make your case.

[#8]

In your data table, which column designates the location?  Have you thought of recoding the written responses (Likert type scale) into an ordinal scale from 1-5?  This may give you more options to look at the data.

[#9]

Hello,

The location variable is rs0.  

For this exploratory and high-level analysis, I had not considered looking at the response as ordinal.  Again, right now, I just want to identify any red flags that I might be able to weed out, or if I can't for some reason (like information loss), I would be prepared to explain the results of my final project objective.  If something were really off, analysis of  T2B and B2B would show it.  For additional perspective, the final activity entails the modeling of the responses to be nominal.

[#10]

You can add the Value Order column property and arrange the strings in a meaningful order. They are ordered alphabetically without this property, which is not significant.