topic Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group in Discussions

Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Fri, 14 Jun 2024 08:48:30 GMT

Hello

I need a little help on picking the right approach to solve my pronlem.

I have a continuous variable X than can take any value between [-45 0].

I have a dataset of X values for 12 distinct groups, see below their histogram.

I'm trying to build two models:

1) A model that, for a given new value of X gives me the probability of belonging to each goup and gives me the most likely group.

2) A model that, for A new entry of X that is suppose to belong to a given group, give me the probablity of this entry not being for that said group.

I initially thaught logistic regression was the best approach, at least for 1) but for 2) I'm not sure. In particular for groups with a very narrow distribution like K for instance.

Thanks in advance!

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MikeD_Anderson — Fri, 14 Jun 2024 12:23:24 GMT

Depending on the data set size, you might have a look at the decision tree methods. Specifically Bootstrap Forest (random forest in the literature).

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Fri, 14 Jun 2024 14:03:14 GMT

Thanks @MikeD_Anderson . Will have a look!

I have about 800 entries but some group have less that 20 observations.

However I just have acces to JMP (not Pro) so I'm still interested in other alternatives with continuous prediction formula. Would ANN work?

Thanks

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Fri, 14 Jun 2024 13:50:36 GMT

Hi @KarlA026 . You may want to try discriminate analysis as well.

https://www.jmp.com/support/help/en/18.0/index.shtml#page/jmp/overview-of-the-discriminant-platform.shtml#

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Fri, 14 Jun 2024 14:02:56 GMT

Thanks @MRB3855 . I get better results with ANN. But still I can help thinking I'm using to sofisticated tools when I have only one (continuous) predictor only.

For approach 2) What about if I fit a distribution model to a group (Normal or other) and use the model formula as a prediction formula?

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Fri, 14 Jun 2024 15:09:44 GMT

Hi @KarlA026 What do you mean by ANN “better” than LR?

And, wrt (2), you are flipping what is random from (1). In (1), group is random. In (2), the way you phrased it, X is random. That’s why I suggested you may want to consider DA.

I’m not sure what you mean by “fit a distribution model to a group (Normal or other) and use the model formula as a prediction formula?”. Can you clarify?

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Mon, 17 Jun 2024 08:24:08 GMT

Thanks @MRB3855 and @MikeD_Anderson

For ANN, I mean that when using X to predict the group, I have less Missclassified results than with LDA. This because LDA just gives a similar model that with logit, not allowing for a non-linear model (see profilers differences in attached images in an example for Group K).

As you can see the profilers for the ANN model is close to the distribution of Group K. My suggestion of fitting a distribution may not be relevant. It’s just that when I tried to fit a normal distribution model to group K for instance, I was able to save the model formula as a new column and use that as an indicator column for any new entry to evaluate the probability of this new entry of belonging to group K.

I just want to estimate, forr a given fixed X value, the probability of that point to belong to a given Group.

Hope I’m clear now!

Thanks!

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Tue, 18 Jun 2024 06:29:33 GMT

@MRB3855 , @MikeD_Anderson,

Any thaughts? Thanks in advance!

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Tue, 18 Jun 2024 08:31:35 GMT

Hi @KarlA026 : Unfortunately, I'm not sure I have a great solution. And, in case one of my comments above flew under the radar, I'll expand on it below.

Above, a few posts up, I said:

"And, wrt (2), you are flipping what is random from (1). In (1), group is random. In (2), the way you phrased it, X is random.".

(1) In Logistic Regression or ANN you are estimating Prob(of being in a group given some value of X), which, for brevity, can be written P(G|X) and read "Probability of Group membership given X". So here, group is the random component. X is fixed (i.e., chosen). So, for a given X, you then estimate the probability of being in each group.

(2) What you "just want to estimate" is very different. The way you've stated the problem, you are asking Prob(X belongs to a group, given a group) = Prob(X|G) read "probability that X belongs to a given group".. Here, X is random and group is fixed. So, what is the probability of X being in a given group. Here, you are choosing the group, and asking what is the probability that X is in that group.

P(G|X) is very different than P(X|G), just like Prob(Person A has the disease, given Person A tested positive for the disease) is very different from Prob(Person A tested positive for the disease, given Person A has the disease).

All said: Looking at your plot at the top, if you choose an x value, say -30 and draw a line straight up. You can see the percentage of area under the curve to the left of -30 for group M is about 50%. For group D it is about 20%, etc. Is that the sorta thing you were thinking when you said “fit a distribution model to a group (Normal or other) and use the model formula as a prediction formula?”

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Tue, 18 Jun 2024 19:07:41 GMT

@MRB3855 thanks for the clarification.
I guess I had that in mind but they I formulated it was not very rigorous.

Anyway, knowing what would you advise to evaluate/model P(G|X) or P(X|G)?

Indeed what you describe in the last paragraph seems close to what I had in mind.

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Wed, 19 Jun 2024 15:25:54 GMT

Hi @KarlA026 : I'm not sure here. Why do you need to "estimate, for a given fixed X value, the probability of that point to belong to a given Group."? i.e., how do you observe X? Is the precise value of X unpredictable (the output of some process that has some uncontrolled source of variability perhaps) or can you select X as you please? And how is this model, whatever the model is, to be used in practice? Can you provide some details?

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Wed, 19 Jun 2024 19:26:41 GMT

Thanks @MRB3855!

I should have start with that. X is an intrinsic material property of a raw material that can be obtained through a specific analysis. The groups are different suppliers of this raw material.

The two scenarios correspond to cases:

1) We have a material for which the supplier is unknown and we need to evaluate the most probable supplier (group) based on X.

2) A supplier of the raw material is provinding a sample and we want to evaluate what’s the probability of this material being effectively form that supplier or, in contrario, the probability of that material coming from somewhere else. All of that based on the value of X for that sample on the reference database shown in the figure.

Hope this helps!

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Thu, 20 Jun 2024 08:04:49 GMT

Hi @KarlA026 : Yes, that is helpful. Thanks!

The idea, as I understand it, is to build a probabilistic model that is based on the data in the figure. That model can then be used to estimate the probability of that sample coming from each of the suppliers, respectively (based on the observed value of material property X).

Case 1 (material from unknown supplier): You measure X. Then once you have the observed value of X, just apply to model you've already built (logistic/ANN/whatever) to predict the most likely supplier (supplier with highest probability).

Case 2 (material from known supplier): If you know the supplier and you know X, the probability of this material being from that supplier (or, not) is irrelevant (though it can be calculated, same as in case 1 above). Here is an example. Let's suppose you look outside and it is raining. But, an hour ago you checked the weather forecast and it said, then, that the probability of rain today is 10%. All you can really say, from a binary prediction point of view, is that the model got it wrong; from a probabilistic view, however, the model wasn't wrong (it didn't say the chance of rain was 0%). But, either way, it's raining...probability doesn't enter into it. So, once something has occurred or is known, there can be no probability associated with it. In your case, once you know the supplier, there can be no probability associated with it...though you can calculate it (just as in case 1 above) to see how well the model predicts. So, if the model predicts supplier M, but you know it actually came from supplier F...all you can say, from a prediction point of view, is that the model got it wrong. That said, do you have some reason to distrust a supplier? e.g., is there a chance you get a sample from supplier D that is actually from supplier B? If so, that is a different situation, with an easy solution (don't use supplier D anymore!).

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

KarlA026 — Thu, 20 Jun 2024 09:43:24 GMT

@MRB3855Many thanks!

To answer your question. Yes in our scenario we assume some raw materials may be coming from other supplier that the one stated (or could be mixed with another source). The aim here is to obtain some kind of certification based on X.

Also, since I have only one predictor here, is ANN an acceptable method? I always thought this was for multivariate issues only.

Re: Best approach and alternative to simple logistic regression to model the probability of belonging to a group

MRB3855 — Thu, 20 Jun 2024 12:34:36 GMT

Hi @KarlA026 : Ah, I see. Good luck!

Certainly, if X = -5 and it is "from supplier D", then I'd be very suspicious based on your plot above. But, what if X = -24? Then perhaps several suppliers are plausible with little ability to accurately discriminate between them. I think X alone is, in general, not a good discriminator. Perhaps there are other material properties?

Sure, ANN is OK; from a purely statistical perspective, I see no reason why you can't use 1 predictor. Though, as I say above, the X you are measuring is, in general, not adequate to discriminate between groups.