topic Re: how to check for Common Method Bias in Discussions

how to check for Common Method Bias

Pradeep_Pai — Wed, 13 Aug 2025 06:46:19 GMT

Please let me know step by step method to perform Common Method Bias in JMP Pro 18

Re: how to check for Common Method Bias

Ross_Metusalem — Fri, 15 Aug 2025 13:49:24 GMT

I'm not previously familiar with common method bias, but some quick reading about it shows that there are multiple techniques for assessing it. The simplest appears to be Harman's single factor "test" (not actually a null hypothesis test), which is just exploratory factor analysis in which you determine if a single factor can account for more than 50% of the variance in your observed variables.

You can do Harman's test easily with Analyze > Multivariate Methods > Factor Analysis. See the documentation for the Factor Analysis platform for general instructions. Specifically for this analysis, you'll want to set the number of factors extracted to 1, then run the model and look at the Variance Explained by Each Factor section. If the Percent value is greater than 50, then according to this method, you have evidence of common method bias.

However, my quick reading suggests that many researchers consider Harman's method to be inferior to methods based on confirmatory factor analysis, which is available in JMP Pro 18's Structural Equation Modeling platform if you want to go that route instead. This example in the documentation demonstrates how to set up a CFA model in the SEM platform.

Re: how to check for Common Method Bias

Pradeep_Pai — Sun, 17 Aug 2025 02:17:14 GMT

Dear Ross,

I did get this very option as suggested by you. However, there is a method of finding the VIF (Variance Inflation Factor), which in SEM is not available. To find VIF, you right click on the parameter estimates, go to Columns, column selection & you should be having the VIF option there. However, in my JMP Pro 18 version the same is not available. I just wanted to know whether this option is unavailable because it is a free academic version? Have a look at what I am saying.

Re: how to check for Common Method Bias

Ross_Metusalem — Mon, 18 Aug 2025 13:31:42 GMT

This isn't a limitation of the academic version (JMP Student Edition). Fit Model can report VIFs in the Parameter Estimates table using the right click menu, but the same option is not present in the SEM platform.

Re: how to check for Common Method Bias

LauraCS — Mon, 18 Aug 2025 14:58:03 GMT

Hello @Pradeep_Pai

As Ross mentioned, there are some alternatives for checking for common method bias. In SEM, you can specify your common factors (latent variables) and then also specify a source of common method bias as another latent variable. The path diagram below is an example of how that would be properly specified:

Here, the scale of the latent variables is set by fixing their variance to 1. In the case of the method latent variable, we also set equality constraints across all its loadings. In this example, we're assuming there's common method across all the observed variables--if you only expect this to happen across a subset of variables, then the loadings can be restricted to just the hypothesized subset. Please see the link that Ross shared for a step-by-step example for how to specify the latent variables.

After estimation, you can square the value of the method loadings to obtain an estimate of common method variance in each observed variable. Relatedly, if the loadings are statistically significant, then you have evidence for common method variance that's not ignorable.

HTH,

~Laura

Re: how to check for Common Method Bias

Pradeep_Pai — Tue, 19 Aug 2025 03:07:26 GMT

Dear Ross,

Please refer to the attached image, obtained from Google Search. It clearly specifies that when we right click on parameter estimates, & select column - column estimates, the option of VIF is available. Either the Google information is incorrect or I am making a mistake. Do note that my requirement of identifying (& rectifying) the common method bias is more than adequately addressed. I have used the Harman's single factor approach for the same.

If you please consider the above & let me know whether in some version the VIF option is available. I had in fact seen a video where the VIF option while doing CFA in SEM was clearly seen. When I went to perform the same in JMP Pro 18, that option was not available.

Re: how to check for Common Method Bias

Pradeep_Pai — Tue, 19 Aug 2025 03:13:03 GMT

Dear M/s Laura,

Thanks for your reply. As mentioned early in my post, I did perform Common Method Bias test & could bring the value below 50%.

In case you wish to know how the CMB can be reduced, please let me know. The only option (mostly) for researchers to collect primary data is survey questionnaire. If the steps, i have identified can be applied then the possibility of CMB is much reduced. All my PhD scholars have to follow these instructions.

Best,

Re: how to check for Common Method Bias

Pradeep_Pai — Tue, 19 Aug 2025 03:17:18 GMT

By the way, thanks to you & Ross for some amazing video explanation of various concepts.

One aspect which is really impressive is that you stick to your timelines in completing the schedule. I am not sure whether you do any editing after recording or are able to do the content - time management in one go. In a live session (& for over 35 years) I have been unable to strictly limit my sessions to the timelines. Either it extends or ends a bit early as the duration depends on discussions & student understanding of the concept.

Re: how to check for Common Method Bias

Ross_Metusalem — Tue, 19 Aug 2025 12:48:18 GMT

Thanks for your kind words about our video tutorials, Pradeep :).

With respect to your Google search, you're not doing anything wrong: Google's AI Overview is incorrect about VIFs in the SEM platform. Google appears to be reporting information from the Fit Least Squares documentation despite your question being specific to SEM. And somewhat frustratingly, when I asked Google the same question (but perhaps with slightly different wording), the AI Overview correctly reported that the SEM platform doesn't report VIFs. So it gives two mutually exclusive answers to the same underlying question! These LLMs are amazing in many ways, but I recommend checking the cited pages whenever the information you've requested is particularly important to you. I've many times found that the AI Overview has made false statements.

Re: how to check for Common Method Bias

LauraCS — Tue, 19 Aug 2025 15:50:06 GMT

Glad you found Harman's test useful! And would love to learn more about the approach you use to handle CMB. I bet others reading this discussion could benefit from it too. I usually try to identify multiple methods of measurement (in surveys, maybe multiple reporters) to deal with this issue. However, that's not always possible.

Best,

~Laura

Re: how to check for Common Method Bias

Pradeep_Pai — Wed, 20 Aug 2025 02:39:29 GMT

Great. I have enrolled for the two-part webinar series for Academics on the use of JMP Pro to build up the course pack. I have also circulated your message across the School (School of Business, Galgotias University) so that my colleagues could also join & register. The only challenge to most of us is the timing of the program, which works out to 9:30 p.m. to 10:30 p.m., our local time. Having said this, I would surely attend the same & send you a Hi!

Re: how to check for Common Method Bias

Pradeep_Pai — Wed, 20 Aug 2025 03:10:39 GMT

Hi!

About 20 to 25 years ago when surveys were conducted without the surveyor (by mail, etc) there was a possibility of the respondent ticking on boxes without even reading the question, in order to complete his / her response (survey bias or acquiescence bias). An inverse scale for some questions, helped overcome this error, which we commonly referred as "satisficing or straight-lining" error.

In the present scenario where multiple items / variables are graded by the respondent, there is a respondent bias resulting in systemic error. Some points listed below helped me reduce the error.

1. More samples in data or sample size > 600

2. More items (variables or questions) in a questionnaire. Though I don't have a threshold for the same, but any survey with questions greater than 25 should suffice.

3. Multi-lingual questionnaire. In India quite a few languages get spoken / understood. If responses are collected in different languages, it helps reduce CMB.

4. Likert scale to be 7 point scale (as against a 5 point scale) for recording responses.

5. Multiple tools of data collection like Google Forms, survey questionnaires, etc.

6. Temporal Separation in completing the survey questionnaire, say part A with 12 questions responded first followed up with part B of the remaining questions after some time delay.

7. Keeping different answers to the same 7 point scale, like let's say most questions have answers ranging from strongly disagree to strongly agree, while some questions have answers ranging from Completely unsatisfactory to completely satisfactory.

I am also attaching here a ChatGPT output which lists pretty much similar steps / observations. I am sure that the next time I run it, it will also add the above points...

Do let me know your views or views of anyone from JMP community.

Re: how to check for Common Method Bias

Ross_Metusalem — Thu, 21 Aug 2025 14:37:40 GMT

Thanks for the suggestions for reducing straight-lining. I don't have a background in survey development so don't have anything to add, but I'm sure some User Community members will find this helpful, and some might have tips of their own to share.

Re: how to check for Common Method Bias

PradeepPai — Sat, 28 Feb 2026 17:16:11 GMT

Hi Ross,

When I tried the single factor EFA, the explained variance is around 55%, which is more than the threshold 50% indicating Common Method Bais. However, when I performed SEM, I got RMSEA 0.092, which being higher than 0.08 indicated SEM is not working. Is there some way that CMB can be reduced?

Re: how to check for Common Method Bias

Ross_Metusalem — Mon, 02 Mar 2026 14:34:19 GMT

@Pradeep_Pai, assessing and remediating CMB in SEM is outside my area of expertise. I'm tagging @LauraCS to see if she has any thoughts.