topic Re: DOE how to estimate the "sample size" of each run in Discussions

DOE how to estimate the "sample size" of each run

anne_sa — Thu, 08 Jun 2023 21:08:38 GMT

Hello everybody,

I have a DOE used to study a response, which is a number of "good" events.

I build a DOE with 20 runs.

I know that on average the proportion of good events is 10%.

During the experiment, for each run, I will apply the corresponding treatment to a sample of plants and count the number of good events among this sample. Finally I will run the analysis using these counts.

I would like to investigate the impact of the sample size associated to each run on the power of the experiment.

I know that there tools to study questions related to the power (using the Evaluate Design or Sample Size Explorer platforms) but I don't really see how to use them in this situation.

Thanks in advance for your help!

Re: DOE how to estimate the "sample size" of each run

statman — Thu, 17 Mar 2022 14:20:11 GMT

Hi, I might not beginning the answer you want, but here are my thoughts:

1. How confident are you the rate is consistent? Are there any patterns or other clues to when the good ones are made?

2. What determines "good" or bad? Can you develop a better response variable? You have pass/fail based on what criteria? Can the criteria be quantified or if the response is a sensory evaluation could you use an ordinal scale?

3. With an average "good" rate of 10%, how much of a change in that rate are you interested in? The smaller the change, the larger the sample size needed. Hopefully you have selected lots of factors with bold level setting.

4. I'm not a fan of sample size calculations...the information you need to know to perform those calculations is usually not available or they are estimates at best. I tend to take a more practical approach. I always consider; Will the amount of samples I take be representative of the situation I am investigating and of future conditions.

You might want to read Bisgaard and Fuller's paper "Analysis of Factorial Experiments with Defects or Defectives as a Response Variable".

Re: DOE how to estimate the "sample size" of each run

Mark_Bailey — Thu, 17 Mar 2022 17:12:08 GMT

You cannot compute the minimum sample size in this case the way you might for something like a one-sample t-test. But you can use simulation, and JMP Pro supports that approach. It requires that you make informed guesses about the effect sizes you might expect and a few other things. See how to use Simulate Responses that works with Custom Design. It sounds as if you can determine the number of events and non-events in each run, so you should use a binomial distribution for response distribution. See how to use the Simulate feature in JMP Pro.

Re: DOE how to estimate the "sample size" of each run

anne_sa — Fri, 18 Mar 2022 10:30:20 GMT

Thanks @statman and @Mark_Bailey for your answers!

I think indeed simulation could be indeed a good tool to explore this question, I will try to play with it.

I will also pay attention to the different points listed by @statman