Discrete Choice Experiments - Adding Weightings to Choices

[#1]

Hi there,

I'm designing a choice experiment and before I collect any responses I want to check if it would be useful to ask respondents how much they prefer one choice over another, as a percentage.

For example, if a respondent chooses Choice 1 over Choice 2, I could then ask them, "as a percentage, how much more do you prefer Choice 1 over Choice 2?" Would collecting this weighting associated with the choice be useful, and if so is there a way to include it in the Choice model platform? I've seen there is  "Weight" column option in the Response Data section of the Choice Model platform.

Many thanks,

Alicia

[#2]

Assume in one response, Choice 1 is chosen. If the percentage, say 200% (2.0), in the response is entered into "Weight", it does not mean Choice 1 is 100% more preferred over Choice 2. It means there are 2 responses chose Choice 1. Just like the person who submitted that response voted Choice 1 twice. There is no comparison information about Choice 2 being entered.

Based on that understanding about what "Weight" does, the percentage information could enter the analysis by constructing the data like this:

1) Duplicate the response, but change the choice of the copy to the opposite choice, Choice 2 in this case.

2) Set a weight to the original response. For this example, it should be proportional to 200%.

3) Set a weight to the copied response. For this example, it should be proportional to 100%.

By such, we are creating a situation, in which the responses that choose Choice 1 are twice as many as the responses that choose Choice 2.

Notice, I said "proportional". Because we need to make sure the sum of two weights is a constant for every pair of responses that are created in this way. E.g. if the sum is 300%, it must be the same 300%  for all the remaining pairs.

But, here, for Choice model, what does 200% preferred mean by a single person? One should either choose or not choose. If the data do not firmly represent choose or not choose, how to interpret the resulting model then?

[#3]

Is it necessary to enter a weight for their preference? Please see the chapter in the JMP documentation about Choice Designs. The associated analysis is based on a logistic regression model for the categorical response. The model will predict the utility, a kind of probability, for different choice sets. I don't mean to say that using a weight is wrong, but I don't know if it will benefit your analysis.

Also, how well do you think respondents can judge such a weight?

[#4]

I agree with Mark.  There are already measurement errors due to the subjective selection of one choice over another.  Adding a scale, especially a percentage scale where there are too many options, would likely increase the measurement error.

You may have options for other response variables.  For example, you might be able to create an ordinal scale (e.g., Likert) that gives a more continuous like response.  Increase the number of choices to more than 2.  Maybe look at MaxDiff experiments? Much depends on the design of the study (e.g., what are the choices, can the choice sets be organized in some rational order, what are you trying to accomplish (explanation or prediction), etc.).