topic more replications or more runs? in Discussions

more replications or more runs?

mahroo — Thu, 27 Jun 2013 02:06:15 GMT

Hi,

I am designing an experiment using custom design in JMP. Number of variables and their levels is high, so I cannot do a full factorial design, but I could have more runs than the minimum that JMP gives me. My question is that which one is better; to have more replications or to have more runs?

Thanks

Re: more replications or more runs?

louv — Thu, 27 Jun 2013 12:31:50 GMT

Your question is not very clear but I can tell you that JMP has powerful diagnostics so you can generate all of the scenarios that you envision and compare the power of your experiments in question with the design diagnostics tools. I utilize the default over the minimum and only choose minimum when material availability is scarce or the experiments are expensive.

Re: more replications or more runs?

mahroo — Tue, 18 Oct 2016 20:14:39 GMT

Thanks for your answer.

Let me clarify my question. My variables are as following:

I have the possibility of having 18 runs. I could set user specified to 18 or I could change the "Number of Replicate Runs" to 4 which result in 18 runs. Here are the result of both approach:

Specifying 18 in "user specified" field:

change the "Number of Replicate Runs" to 4:

Could you please let me know how to decide based on these results.

Thanks!

Re: more replications or more runs?

louv — Tue, 18 Oct 2016 20:14:55 GMT

They look very comparable. When I ran the algorithm and giving it 1000 starts under the red triangle my results were lightly different.

I wouldn't lose sleep over either choice.

How about a compromise with 2 replicates and 18 runs total? I have attached all three scenarios with the same simulated response model for you to see. Just run the Fit model scripts.

As you can see all approaches are comparable.

Re: more replications or more runs?

mahroo — Thu, 27 Jun 2013 16:17:33 GMT

Thanks!

Is it possible for you to give me an example that one approach is preferable to the other one?

Re: more replications or more runs?

louv — Thu, 27 Jun 2013 16:30:36 GMT

Not really. Of the three I prefer the compromise with 2 replicates and 18 runs however quite honestly I'm more concerned that you have not included two-way interactions in the experiment.

Re: more replications or more runs?

mahroo — Thu, 27 Jun 2013 16:47:20 GMT

Ya, I did not include two-way interactions since it increases the number of runs.

Re: more replications or more runs?

louv — Thu, 27 Jun 2013 17:15:31 GMT

Do you expect interactions from your system? The true power of DOE comes from uncovering interactions which allows you to tune out sensitivities in your system. Since I do not know the context of your problem it is difficult to be much more helpful. Yes the interactions in your model would require more runs and since your factors are all categorical the number of experiments grow quickly. Are any of your categorical factors candidates to be treated in continuous fashion? If so the number of runs can be more manageable.

Re: more replications or more runs?

mahroo — Thu, 27 Jun 2013 17:24:46 GMT

My experiment is actually designing intersections in driving simulator. each run setting will be an intersection, and each variable is associated with a condition of the intersection (for example variable "pavement condition" with two levels of "wet" and "dry". It is good to have two-way interactions, but as we hire drivers, more number of runs would be expensive.

No, none of my factors could be continuous.

Re: more replications or more runs?

louv — Tue, 18 Oct 2016 20:15:04 GMT

You could always treat the two-way interaction model terms as "if possible" for the same investment of runs.

Re: more replications or more runs?

mahroo — Thu, 27 Jun 2013 17:47:07 GMT

That's a good point, but I guess if it does not have any effect on the design of experiment I could consider it later when I will be working on analysis of the results.

Re: more replications or more runs?

louv — Thu, 27 Jun 2013 18:02:47 GMT

Although it is the same number of runs if you look closely at the attached file it does have an effect on the design since the algorithm gives a different set of runs. You sacrifice some power on the main effects to be able to possibly model the interactions. You are banking on the effect sparsity principle and by utilizing a stepwise regression you are hoping to "tease" out a more complex model.

Re: more replications or more runs?

mahroo — Thu, 27 Jun 2013 18:28:24 GMT

Thanks for all your help