Two alternatives in choice experiment

[#1]

Hi, I am designing an experiment with two discrete choice options, 2 factors each with 2 levels, and 1 with 6 levels. I have tried DOE consumer studies, choice design but unable to make this work. Any help will be greatly appreciated. Thanks.  

[#2]

Hi Victor, Yes table 2 is a transpose. Does this study design make sense? Also, I did try to estimate the simulated data and it gave me all zeros. Any thoughts please? Thank you so much for following up on these two questions. Regards. 

[#3]

Hi @aus22,

Can you provide more context and infos ? JMP file and the steps you have done leading you to 0 ?

Using your Excel file and the Choice simulated column, I have a "normal" analysis window :

You may encounter "a lot of zeros" for the factors estimates if your response is random binary 0/1 and not influenced by specific factors. Also if you are trying to fit a complex model to a small and simple design, you can obtain zero on logworth, so no significant p-values :

In this case, the correlations between terms in the model are high, preventing the precise estimation of effects and inflating std error. It's becoming almost impossible to evaluate statistical significance in this situation, so p-values are equal to 1 for each term.

Attached please find your file.

Hope this answer may help you,

[#4]

Hi Victor, I understand, thank you. I am still unclear on whether the designs I have generated are correct or not. Can you please help with the enclosed file:

Choice Profiles - Part 1 Oct. 15 - This design's objective is to understand the factors that will influence respondent choice between two alternatives with three attributes. 

Choice Profiles - Part 2 Oct 15. - This design's objective is to understand the factors that will influence respondent choice between 4 products with those similar factors except we need an additional one that specifies the type of materials used in the product. We need to offer 4 alternatives given the 4 different types of materials that can be used to create this product. 

Can you please advise if these designs are ok? Thank you so much. 

[#5]

Hi @aus22,

It may be difficult to assess if the designs are acceptable for your needs, as I don't know your industry, context, topic, etc...

However, some questions that may help :

• In your two designs, I don't see information about the respondents. Do you only have one ? If yes, how sure and confident are you that this person will be consistent, reliable, accurate and representative of your consumers (or studied population) to discriminate the choices ? Having several respondents may also help understanding the generalization of the outcomes and conclusions, regarding a particular benefit of one of the factor level in the choices.
• Are the products between the two studies the same ? If yes, why not creating a design for part 1 with also the material type ? Including this factor early in the design could help discriminate the material type quickly, and understand its relative importance compared to other factors.
• Think iteratively. I think using a first design as "screening" design might be helpful, but you can augment this design and use the prior information generated from the first design to confirm and precise the relative importances of factors, estimates, utility, etc... You can find an iterative example here : Design the Final Choice Experiment Using Prior Information 
  Also using an iterative approach instead of a "one-shot" design is helpful to manage expectations, as you can always augment and enrich your understanding and knowledge about the products.

All the best for your experimentations !

I hope these few remarks may help you to solve your problem,

[#6]

Hi Victor, thank you so much for your thoughtful advice. I actually did not include the no of respondents in the info I sent. We plan to collect data from at least 200 respondents. I like your suggestion of thinking through an iterative approach. 

JMP question: One challenge I am facing is that I am unable to get the software to provide a design for only one survey with that number of parameters. So would it be ok for me to use Survey 1 and Survey 2 in my enclosed designs to be randomly assigned to each of my 200 or so respondents? Thank you as always. 

[#7]

Hi @aus22,

Related to your answer and JMP question :

If you have 200 respondents, I would consider providing several random surveys to random group of people, instead of one survey for all participants. Having different choices sets in different surveys enable to test different combinations in each survey, and help generalize the findings and better estimate relative importance of the factors.

It's a compromise to find : avoid having a specific survey for each participant (which could lower the precision of estimation of the factors relative importance), but avoid having one survey for all participants (which could compromise generalization of the results as less combinations are tested, and could create some noise due to inconsistent results from some participants).

As an example, you could maybe create 4-5 surveys for each topic, so that you collect results from groups of 40-50 participants ? 

[#8]

Hi Victor, thank you for that suggestion, I appreciate it a lot. I have never done multiple surveys for the same respondent so not very confident of how to develop the survey instrument, and do the estimation. If you have any resources from JMP that you can share I will truly appreciate it because what you are suggesting does sounds like a better approach. Thank you!! 

[#9]

Hi @aus22,

To be sure there is no misunderstanding, the suggestion I made was to create 4-5 different surveys for 4-5 groups of different (random) respondants, not that each respondants answer to multiple surveys.

Here is a snapshot of what I meant in terms of parameters in the Choice design platform : 

When creating this design, you'll have 200 independent respondants, distributed in 4 groups of 50 respondants, each group answering a specific survey. Please find attached the design used here as an example, to see the repartition of 200 respondants in 4 groups and the combinations and choices set used in each survey. The platform "Choice" for the analysis takes into account the survey number and respondent ID, so analysis is quite straightforward and simple, and already prepared with the "Choice" script present in the data table generated by the design creation platform.

I hope this response will help you,