topic Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneously in Discussions

Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneously

phildias — Fri, 17 Aug 2018 21:08:35 GMT

Hi there,

I'm designing a survey where people need to choose one of three transportation alternatives. Here are the alternatives, their specific factors and the number of levels I intend to use for each

Mode A:
- Fixed cost: 3 levels: $10.00, $15.00, $20.00
- Variable cost: 3 levels: $0.25, $0.50, $0.75
- No waiting time
Mode B:
- Fixed cost: 3 levels: $5.00, $15.00, $30.00
- Variable cost: 3 levels: $0.25, $0.50, $0.75
- No waiting time
Mode C:
- No fixed cost
- Variable cost: 3 levels: $2.00, $3.00, $4.00
- Waiting time: 3 levels: 2min, 5min, 8min

Besides all of this, there is another factor that should be taken into account in the design, which doesn't "belong" to any of the alternatives. It would be something like this:

Weather condition: 3 levels: sunny, rainy and overcast

Notice how I would have ONE weather condition per choice occasion, not one weather condition PER ALTERNATIVE.

On top of this design, I would also like each respondent to receive only 2 choice occasions (i.e. 2 choice sets per survey).

So, for example, a survey would look somehting like this:

Question 1

When the weather is sunny, which one of these modes would you use?

Mode A: FC=$10.00, VC=$0.75, WT=0min

Mode B: FC=$30.00, VC=$0.50, WT=0min

Mode C: FC=$0.00, VC=$3.00, WT=5min

Question 2

When the weather is rainy, which one of these modes would you use?

Mode A: FC=$15.00, VC=$0.50, WT=0min

Mode B: FC=$5.00, VC=$0.75, WT=0min

Mode C: FC=$0.00, VC=$4.00, WT=2min

It would be great to have around 10 versions of the surey. If that's the case, I expect to have about 100 respondents per survey.

Is there any way to do this in JMP? All I have seen in the "Choice Design" module is for alternatives that share EXACTLY the same attributes and levels. I also haven't found where to indroduce "outside" variables that don't belong to any particular alternative, but still affect people's choices (such as Weather condition).

How would I go about building an experiment like this using JMP?

Thank you!

Kind regards,

Felipe

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Mon, 20 Aug 2018 17:22:06 GMT

Bumping this up to get more attention!

Thanks!

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

txnelson — Mon, 20 Aug 2018 15:54:29 GMT

I would give this topic more time. You asked your question on a Friday, and many users, or JMP Staff have had a weekend where they probably have not been on the Community Website.

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Mon, 20 Aug 2018 16:03:56 GMT

Ah, okay! Sorry about that. Thanks, then, will do! I was just getting sad seeing the question float by and getting pushed past the first few pages of the question board, hahahaha...

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Wed, 22 Aug 2018 18:20:39 GMT

One more bump, trying to get this thread some more attention! =)

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Fri, 24 Aug 2018 16:36:06 GMT

At this point, given the lack of a response after a week, would it be safe to say JMP can't deal with this kind of design?

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

Ryan_Lekivetz — Fri, 24 Aug 2018 17:37:09 GMT

Hi Felipe,

Apologies that I hadn't seen this earlier.

For cases like this, I usually end up finding some type of solution in Custom Design to create a choice design, where one set of random blocks (very hard to change) represents the surveys, and another the choice sets (hard to change). See one example here: https://community.jmp.com/t5/JMP-Blog/Potato-chip-smackdown-US-vs-Canada/ba-p/30641

Because the modes have their own set of restrictions, you could try to use disallowed combinations, but instead I found it easier to set it up where each row represents a choice set, and assume all 3 modes are given in each choice set. So I put 10 for Number of Whole Plots to represent the number of surveys, and 20 runs, which specifies that each survey will have 2 rows (i.e. 2 choice sets per survey). Below is the jsl to set this up.

Hope this is on the right track to helping you.

Cheers,

Ryan

DOE(
Custom Design,
{Add Response( Maximize, "Y", ., ., . ),
Add Factor( Categorical, {"sunny", "rainy", "overcast"}, "weather", 0 ),
Add Factor( Categorical, {"10", "15", "20"}, "Acost", 0 ),
Add Factor( Categorical, {".25", ".5", ".75"}, "Avar", 0 ),
Add Factor( Categorical, {"5", "15", "30"}, "Bcost", 0 ),
Add Factor( Categorical, {".25", ".5", ".75"}, "Bvar", 0 ),
Add Factor( Categorical, {"2", "3", "4"}, "Cvar", 0 ),
Add Factor( Categorical, {"2", "5", "8"}, "Cwait", 0 ),
Add Factor( Continuous, -1, 1, "fake factor", 1 ), Set Random Seed( 448800 ),
Add Term( {1, 0} ), Add Term( {1, 1} ), Add Term( {2, 1} ), Add Term( {3, 1} ),
Add Term( {4, 1} ), Add Term( {5, 1} ), Add Term( {6, 1} ), Add Term( {7, 1} ),
set N Whole Plots(10), Set Sample Size( 20 )}
);

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Fri, 24 Aug 2018 21:35:33 GMT

Hi Ryan,

Thanks so much for the reply! It definitely helps a lot and is a great starting point!

I do have a couple of questions, though.

Firslty, the code you sent seems to be treating the outcome as a regular continuous outcome, right? Is there any way create this design within the confines JMP's "Choice DoE" module? I ask mainly because of this blog post, where the author makes a strong case against using regular industrial (i.e. continuous outcome) experimental design tools to develop experimental designs for discrete-choice outcomes.

Also, is there any particular reason why you used "Categorical" instead of "Discrete Numeric" for the cost and time factor levels? It seems like the latter might be more appropriate. I'm not sure if it genuinely makes a difference in this case, so I just wanted to ask.

Regarding the multiple runs, I tried playing around with the Custom Design module and noticed that I could use the "blocking" type-variable to represent the multiple runs/choice occasions for each respondent. So what's the difference between this line that you sent:

Add Factor( Continuous, -1, 1, "fake factor", 1 ), Set Random Seed( 448800 ),

and how I represented the choice-occasions here?

Add Factor( Blocking, 2, "Blocking" ), Set Random Seed( 1970753291 ),

Lastly, just to be clear, "Whole Plots" represents the number of versions of the survey, while "Runs" (or "Sample Size") represents number of surveys * number of choice occasions?

Again, thanks so much for the help!

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

Ryan_Lekivetz — Tue, 27 Nov 2018 18:26:48 GMT

Great questions!

The Custom Design will indeed be optimized for a linear model, and not the Choice model. It really is for special cases like this, where the Choice Designer can't handle the setup. What you'll typically find is that the Custom Designer does a reasonable job of spreading out the different factors among surveys/choice sets. It won't be able to address some of those points mentioned in the other blog - sometimes you have to do this manually, and sometime you can still do a bit of it in the Custom Designer using disallowed combinations (https://community.jmp.com/t5/JMP-Blog/Using-the-Disallowed-Combinations-Filter-in-JMP-12/ba-p/30588), although this can be tedious if you put a lot of restrictions.

For the Discrete Numeric, the biggest difference is that when the Designer is treating it as numeric instead of categorical, it's not uncommon to see the middle level (which then needs to be specified with a quadratic term) not appear as often as the high and low. Really more of a design decision than anything.

As for the blocks, you could used fixed blocks, but you may run into trouble, because it's using one degree in each block to estimate a block effect. Based on the way it's specified, it may end up complaining it can't fit the model. The random block approach will give you more flexibility in that respect ( https://community.jmp.com/t5/JMP-Blog/Benefits-of-experiment-design-using-random-blocks/ba-p/30166 )

Lastly, just to be clear, "Whole Plots" represents the number of versions of the survey, while "Runs" (or "Sample Size") represents number of surveys * number of choice occasions?

Correct - note that you could also take the approach where each run is a question, so you have an extra multiplier (like the potato chip example), but it would require some extra work because of the way each mode is structured.

Hope this helps,

Ryan

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

phildias — Mon, 27 Aug 2018 16:51:01 GMT

Awesome, thank you so much for the clarifications!

Re: Choice DOE with alternative-specific factors and factors that affect all alternatives simultaneo

karlim — Tue, 22 Oct 2019 17:44:27 GMT

Is it possible to add Bayesian priors into this design?