topic Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression in Discussions

Contradictory Ranking: Group Means vs Nominal Logistic Regression

tomSorger — Fri, 09 Jun 2023 00:56:02 GMT

We have four categories, each with 1000 (continuous) responses. The order for the average response A < B < C << D. The order obtained by nominal logistic regression: A < C < B << D. (The difference between means B and C is 1.64%). I understand that nominal logistic regression is based on likelihood ratios for different rankings, but it is still puzzling to me that the same dataset yields a different order when the dependent and independent variables are reversed i.e. response as f (category) versus category as f (response).

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

peng_liu — Tue, 25 Oct 2022 19:47:21 GMT

I am not sure the meaning of "The order obtained by nominal logistic regression", because"nominal" means no order.
But in general, you have two models here. Regardless how you draw conclusions from the two, they don't have to agree. But you need to validate each of them to decide which is more trustworthy, or neither.
One question: whether the difference between B and C significant in one model. If not, there is nothing to debate, because the order is inconclusive according to that model.

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

tomSorger — Thu, 27 Oct 2022 13:48:00 GMT

Thank you for your helpful comment.

Here is another example from a similar dataset (250 responses per category).

Sample means: A < B < C << D. (D is significantly higher than A or B).

Nominal logreg: B < C < A < D.

Nominal regression orders the categories without ordinal information.

So for the category rank, when responses are the dependent variable, we get the first ranking.

But when we try to obtain the category order based on the responses, we get the second ranking.

It is not apparent to me why these two models should disagree.

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

peng_liu — Thu, 27 Oct 2022 22:38:30 GMT

You have not explained what you mean by "the order obtained by nominal logistic regression". Here is my guess, do you mean the order appeared in the Logistic Plot as in the following screenshot?

If that is the case, you have a misconception about this plot. Those labels are ordered alphabetically. As I mentioned earlier, "nominal" means no order. If you conclude some kind of order out of the Nominal logistic regression, I believe that must be some probability measure. And I don't see your case is such a type.

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

tomSorger — Fri, 28 Oct 2022 13:46:55 GMT

Thank you very much for correcting my misconception. I have one follow-up question, if I may. If we know the ranks of the means, is it valid to perform an ordinal logistic regression based on those ranks? Thanks again,

Tom

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

Jed_Campbell — Fri, 28 Oct 2022 15:02:37 GMT

If we know the ranks of the means, is it valid to perform an ordinal logistic regression based on those ranks?

I'm not sure I understand what you're asking, but ordinal logistic regression is generally only a valid approach when imposing an order on the values is logical before running the model, i.e.:

Small, Medium, Large: ordinal logistic regression is appropriate
Supplier A, Supplier B, Supplier C, Supplier D: ordinal logistic regression is not appropriate

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

tomSorger — Fri, 28 Oct 2022 17:54:35 GMT

Thank you for the reply.

I already know the order of the category means:

A < B < C < D.

Q1: Now, if I run an ordinal logistic regression, does the P value

for the likelihood ratio tests provide useful added information

beyond ANOVA?

Q2: As an alternative to ANOVA, is it valid to compare the P values

among all the possible rankings? I would like to better understand

why this is, or is not, valid.

Thank you, these are my final questions!

Tom

Re: Contradictory Ranking: Group Means vs Nominal Logistic Regression

peng_liu — Fri, 28 Oct 2022 20:07:01 GMT

It might be useful if you can actually post your analysis, so we can know better about what we are talking about. It can also be useful if you can state the problem of interest, or hypothesis.

Meanwhile, in case the following general comments still make sense to you, I am trying a different way to see whether it can help you to understand the problem in your hands.

For Q1, here is an example using the Iris data among JMP sample data. This is a logistic regression of Species ~ Sepal Length. What the p-value that I highlighted in the screenshot means: Sepal can tell there are differences among three categories. But that is it, nothing else.

Further down the report, there is this estimates report. The p-values here indicate that setosa-vs-virginica and versicolor-vs-virginica are separable. But that is it, nothing more. No orders.

For Q2, I am not sure which report that you refer to. Following is a screenshot of "All Pairs, Tukey's HSD"

To interpret this report, you may want to check out this documentation: All Pairwise Comparisons This documentation is under Standard Least Squares platform, but the results are equivalent to what are in Fit Y by X.(numbers might be different by signs, due to ordering difference.)

Please read the first paragraph of the documentation carefully. When you compare pair by pair, different statistics have different meanings. In particular, you should look for words like "pairwise" and "joint". This is related to multiple comparisons. And you may want to read this wiki page to understand why the words matter: Multiple comparisons problem