topic Re: How do I correct the cells that have an expected count less than 5? in Discussions

How can I do a chi-square test when 20% of the cells that have an expected count less than 5?

slamer2000 — Wed, 10 Jan 2018 18:21:36 GMT

I've got these warning messages (see below) after doing some analysis with some variables. So to fix this problem, I read in a stats book that I need to follow some steps that it stated in order to correct the problem such as "If the smallest expected cell frequency is less than five, the analyst should recode a row or column variable, combining rows or columns until an adequate expected cell frequency is achieved."

However, I don't really understand this or how to do it in JMP. Can someone explain how I might change modify things so that I won't get his warning message either through the instructions from the book I've just mentioned or from a better way? I've also heard that I one can alternatively use the fisher's test?

Thanks!

Re: How do I correct the cells that have an expected count less than 5?

txnelson — Sat, 06 May 2017 15:02:58 GMT

I made a quick scan of your results( which means my statistical viewpoint may be less than accurate), it appears that few respondants have answered "Disagree" or "Strongly Disagree". So you may want to do one of 2 things. Either eliminate those respondants, or recode their answers.

To eliminate those responses, you can open up a local data filter by going to the red triangle at the top of the display, and selecting Local Data Filter. There you will be able to select only the data you want. You can also go to the data table, Select all rows that match either of the choices, and then right click on the row state column for one of the selected rows, and select "Hide and Exclude". When you rerun the analysis those rows will be left out of the analysis.

If you want to recode, then go to the column in question, "Recycling can.........", select that column, and then in the Columns Panel at the left of the data table click on the red triangle and select Recode. The window that pops up will let you combine the Disagree and Strongly Disagree or maybe to take both of these choices and move them into Neutral

Re: How do I correct the cells that have an expected count less than 5?

slamer2000 — Sun, 07 May 2017 11:04:41 GMT

But if I eliminate those responses, won't that also eliminate their data from my analysis which would alter the results since I'm excluding some responses? I'm not really sure how eliminating respondants fixes the problem; which respondents should I eliminate? Could you explain how doing either these two might fix my problem as I'm having a hard time understanding the concept. Because from what I see, if I combine columns, then how about the data within the columns that had a few responses? Isn't that information useful even if just one person selected that option such as the "I don't know" option? Thanks!

Re: How do I correct the cells that have an expected count less than 5?

txnelson — Sun, 07 May 2017 13:08:32 GMT

Your concerns are valid. And what I illustrated is a method to allow you to collapse or eliminate data so that you can meet the requirements of the statistical test. But as you indicate, that method comes with a cost.

Re: How do I correct the cells that have an expected count less than 5?

slamer2000 — Sun, 07 May 2017 13:56:35 GMT

So is there any way to fix this issue without altering the data significantly? Or will the methods that you mentioned not significantly affect the results?

Re: How do I correct the cells that have an expected count less than 5?

txnelson — Sun, 07 May 2017 14:34:19 GMT

The data are, what the data are. The statistics are based upon the relationships with between the different cells. You may be able to overcome the issue by increasing the number of data points, but that would require getting more subjects. Any concatenations, or eliminations that you choose to do, need to be evaluated from the real world. Many times, the very small cell counts are meaningless because of the low number of responses. In those cases, you really can not draw any opinion on those results anyway. So even if you left them in, I would not want to make any statement on those cells anyway. Taking them out, therefore, may allow you to show stronger relationships with the remaining data.

Re: How do I correct the cells that have an expected count less than 5?

Mark_Bailey — Sun, 07 May 2017 21:13:39 GMT

The Chi square test used in the Contingency platform requires at least 80% of the cells to have an expected count greater than 5 or else the sum of the cell Chi squares will not have a Chi square distribution and so your test (p-value) will not be valid.

The usual, simple solution in such cases is to combine levels. For example, in your case you might replace the five levels with just three (Disagree, Neutral, Agree) and see if your assumption is valid. This way does not exclude any data from the analysis.

Re: How do I correct the cells that have an expected count less than 5?

slamer2000 — Mon, 08 May 2017 08:30:36 GMT

Thanks you both for your answers, I found them to be very helpful. But I found another problem in other parts of my data. Your solution to combine levels and to have just 3 levels does work in some circumstances. But in the below results, I don't think I can do this as you can see there are just too many cells that have zero expected values in multiple levels, making it hard to combine them. I think then, the Fisher's exact test is the last resort that I have to fix this problem. But I would like to ask how can I do the Fisher's test in JMP when my variables have multiple categories resulting in tables that have numerous rows/columns (e.g 4x5, 5x5) while Fisher's exact test requries a 2x2 table?

Re: How do I correct the cells that have an expected count less than 5?

Mark_Bailey — Mon, 08 May 2017 11:51:06 GMT

First of all, a clarification: the validity is not based on zero counts, but cells with expected counts less than five.

JMP provides the exact test only for 2x2 contingency tables. JMP Pro or SAS/STAT provides the exact test for larger tables.

Re: How do I correct the cells that have an expected count less than 5?

slamer2000 — Mon, 08 May 2017 16:06:21 GMT

Yes, I am aware that the validity is based on cells with expected count less than 5, I just highlighted those cells to show why I wouldn't be able to combine the levels since there were cells with expected count less than 5 on both the neutral and the disagree levels.

Well it is unfortunate that I would need to get those software in order to do this test. So is there any other solution for my problem?

Re: How do I correct the cells that have an expected count less than 5?

Mark_Bailey — Mon, 08 May 2017 17:27:47 GMT

Let's see what we can do.

First of all, I believe that you have the Y and X roles reversed. The responses (e.g., Neutral) should be in the Y role (levels running across the top of the Contigency table). Second of all, your example of a predictor variable X is ordinal. It is a ranking of decreasing cycling activity. As such, you should (1) apply the ordinal modeling type and (2) add a Value Ordering column property. For example, in the example that you posted, the first level shown should likely be the last level after Less Than Once a Week. You might be able to take advantage of the ordinal nature of X and Y and use a more powerful association test than the contingency table Chi square.

The fact that you are getting no or just a few responses in adjoining levels means that you can collapse them into a single level. The example you show has essentially all positive mass: it exhibits 8 neutral or negative responses out of 190. You have a lot of observations but spreading them out over 25 cells means that you are likely to fail the validity check.

Re: How do I correct the cells that have an expected count less than 5?

slamer2000 — Thu, 11 May 2017 14:41:02 GMT

Thanks Mark for sticking with me so far! I've tried collapsing the levels together and doing all the things you mentioned. And silly silly me, I realized that I've been looking at the actual counts, forgetting that they're not the same as expected counts, which is what you were saying earlier, that validity is based not on zero counts but expected counts less than 5. I already knew this but it just went over my head. Anyways, sorry about that. Looking now at my results after making all the changes and adjustments you suggested, it seems with my data, theres always this one (or two) lingering cell(s) that have expected counts less than 1 like in the below example (Figure 1).

I found this guideline at a website: "If both variables have 4 to 6 levels, then you can trust the results if either of the following is true:

1. All cells have expected counts of at least 2.

2. All cells have expected counts of at least 1, and 50% or fewer of the cells have expected counts of less than 5."

And so I'm hoping to at least satisfy the second point but as you can see, one of the cells under Neutral/D/SD, doesn't quite cut it. In the end though, after getting your suggestion (as well as others from forums) to use other association tests for ordinal data, I'll try to experiment using other tests suited for ordinal data if theres not much more I can do. But I'd still like to get your thoughts about whether theres anyway I might be able eventually utilize the chisquare method for this data. Unless, you strongly suggest that I just move on from this and use association tests for ordinal data since it would give better results.

Also, I'm not exactly sure how to do ordinal measures of association on JMP but I'm guessing that when you use the Fit Y by X function, it automatically does all these tests for you depending on what your data is like. As I found when I clicked the orange tab button next to "Contingency.." and selected Measures of Association, it showed me all these outputs for a lot of different tests (figure 2), some of which I recognized as tests for ordinal data like Somers'd. Is this how I find the analysis results for different tests without having to manually select the right test like how SPSS would require you to do?

Figure 1Figure 2