Our World Statistics Day conversations have been a great reminder of how much statistics can inform our lives. Do you have an example of how statistics has made a difference in your life? Share your story with the Community!
Choose Language Hide Translation Bar
Highlighted
Level I

## more replications or more runs?

Hi,

I am designing an experiment using custom design in JMP. Number of variables and their levels is high, so I cannot do a full factorial design, but I could have more runs than the minimum that JMP gives me. My question is that which one is better; to have more replications or to have more runs?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Staff (Retired)

## Re: more replications or more runs?

They look very comparable. When I ran the algorithm and giving it 1000 starts under the red triangle my results were lightly different.

I wouldn't lose sleep over either choice.

How about a compromise with 2 replicates and 18 runs total? I have attached all three scenarios with the same simulated response model for you to see. Just run the Fit model scripts.

As you can see all approaches are comparable.

12 REPLIES 12
Highlighted
Staff (Retired)

## Re: more replications or more runs?

Your question is not very clear but I can tell you that JMP has powerful diagnostics so you can generate all of the scenarios that you envision and compare the power of your experiments in question with the design diagnostics tools. I utilize the default over the minimum and only choose minimum when material availability is scarce or the experiments are expensive.

Highlighted
Level I

## Re: more replications or more runs?

Let me clarify my question. My variables are as following:

I have the possibility of having 18 runs. I could set user specified to 18 or I could change the "Number of Replicate Runs" to 4 which result in 18 runs. Here are the result of both approach:

Specifying 18 in "user specified" field:

change the "Number of Replicate Runs" to 4:

Could you please let me know how to decide based on these results.

Thanks!

Highlighted
Staff (Retired)

## Re: more replications or more runs?

They look very comparable. When I ran the algorithm and giving it 1000 starts under the red triangle my results were lightly different.

I wouldn't lose sleep over either choice.

How about a compromise with 2 replicates and 18 runs total? I have attached all three scenarios with the same simulated response model for you to see. Just run the Fit model scripts.

As you can see all approaches are comparable.

Highlighted
Level I

## Re: more replications or more runs?

Thanks!

Is it possible for you to give me an example that one approach is preferable to the other one?

Highlighted
Staff (Retired)

## Re: more replications or more runs?

Not really. Of the three I prefer the compromise with 2 replicates and 18 runs however quite honestly I'm more concerned that you have not included two-way interactions in the experiment.

Highlighted
Level I

## Re: more replications or more runs?

Ya, I did not include two-way interactions since it increases the number of runs.

Highlighted
Staff (Retired)

## Re: more replications or more runs?

Do you expect interactions from your system? The true power of DOE comes from uncovering interactions which allows you to tune out sensitivities in your system. Since I do not know the context of your problem it is difficult to be much more helpful. Yes the interactions in your model would require more runs and since your factors are all categorical the number of experiments grow quickly. Are any of your categorical factors candidates to be treated in continuous fashion? If so the number of runs can be more manageable.

Highlighted
Level I

## Re: more replications or more runs?

My experiment is actually designing intersections in driving simulator. each run setting will be an intersection, and each variable is associated with a condition of the intersection (for example variable "pavement condition" with two levels of "wet" and "dry". It is good to have two-way interactions, but as we hire drivers, more number of runs would be expensive.

No, none of my factors could be continuous.

Highlighted
Staff (Retired)

## Re: more replications or more runs?

You could always treat the two-way interaction model terms as "if possible" for the same investment of runs.

Article Labels

There are no labels assigned to this post.