How to treat replicates and failed runs in a definitive screening design

[#1]

Hi community, I am having trouble modeling some data that I got from a DSD. This was a screen of 10 factors. We ran this with 3 blocks (operators) and I included the maximum number of extra runs (8). This gave me a table of 31 runs. We did some initial range-finding experiments to set reasonable bounds for all of the factors. Unfortunately we still got 4 runs where the reaction completely failed, so I have no value to put in as a response.

My first question is how these failed runs should be treated? Ideally I suppose they should be excluded, and I would still have 27 runs that did provide data. But if I exclude them from the table and try to run the Fit Definitive Screening script, nothing happens. There is no error message but the window just doesn't open. My response is "Time to Completion", with the objective of minimizing this response, and I could just put in a large number representing a time longer than the experiment duration but this seems arbitrary. The script will run though.

My second question is whether I can include replicate runs to have more confidence in the model? We ran 8 replicates of each run condition. However, if I copy and paste a run and try to run the script, again nothing happens. Right now I am just taking the average of the 8 runs and using that, but I'm wondering if it would be better to somehow include the individual values and if so, is there a way to do it with the Fit Definitive Screening script?

I can use the standard Fit Model module to solve both of these problems but it seems like I'd be missing the benefit of the DSD. Thanks for any responses! I'm on JMP 15.2. If posting the table would be helpful I'm happy to do it.

[#2]

My thought process when analyzing DOE data is to start with a saturated model. This is called the subtractive approach to model building (vs. the additive approach like stepwise does). A saturated model is when all degrees of freedom are assigned. In your case, I treated the block as a fixed effect and include block-by-factor interactions.  When analyzing the results first think what practical effect does each factor (model term) have and then look to statistical significance.  Remove the insignificant items, then re-run the reduced model and analyze the residuals.  I don't know what practical significance is for your data set, so I could not reduce the model.

[#3]

Thanks statman, I think I understand the approach. 

To generate my saturated model, I have been starting with all factors, all factors squared, and all interactions (using the Response Surface macro). Then running a Standard Least Squares. Is there an option for treating the block as a fixed effect? Currently the result lists a couple singularities that seem to be preventing me from getting p values for any of the interactions. Getting stuck there. In your model, you have all of the factors and a few interactions but not all of them, does this mean that you have reduced the model somehow already? How did you know which ones to remove?

[#4]

In the future, I would recommend knowing what model effects you want to separate, which model effects you are willing to confound and which model effects you don't want to include in the study BEFORE selecting the design to run!

You can't use the response surface macro and you must ensure the block is not a random effect.  To create the model, you will need to understand the total degrees of freedom available and the aliasing in your experiment.  (Never put aliased terms in the model, you will get singularity errors).  You might start here:

https://www.jmp.com/support/help/en/16.2/?os=mac&source=application&utm_source=helpmenu&utm_medium=a...

When you create the design in JMP, you should get an arrow that is a script to Evaluate Design, this will provide the aliasing structure.  If you have run a complete block, your model will contain the definitive screening design model, block and all block-by-definitive screening model effects.

 

[#5]

The more I learn, the more I realize how much I don't know. Thank you for the continued guidance!

[#6]

My pleasure.  Experimental design is a huge topic with many "facets".  What further complicates the subject, is there is no one "right way".  Some will be more efficient, some will be more effective.  I have found that I have to first understand the situation, what questions I'm trying to answer, what hypotheses I'm trying to get insight into, what model effects do I need to estimate. I design multiple options/plans, think through what knowledge could be potentially gained (I actually predict every possible outcome) for each option and weigh that against the resource requirements.  Recognizing that I will be iterating, so what do I need to continue the investigation.

I have been extremely fortunate to learn from some of the best minds (Box, Taguchi, McLean, Sanders, Diamond, Bisgaard, et.al.) and had the luxury of being involved in designing and analyzing 1000's of experiments.  Many of witch in retrospective could have been designed better...LOL

[#7]

Hi @Mark_Bailey 

Following up on that, how do you input the individual replicates in the DSD? When designing the table of the DOE, there is no box to indicate your number of replicates (while this box is present when designing a classical screening design for example).

Thank you

Sara

[#8]

You do not have the same option with a DSD. You cannot replicate some runs if you want to use Fit Definitive Screening. You can copy all the runs after you click Make Table, and paste a copy below the original rows to replicate the design.

[#9]

Hi @SaraA,

As @Mark_Bailey has mentioned, you can replicate manually the full design by copy-pasting all the rows belonging to the DSD.
You can also use the platform Augment Design to replicate your DSD : Replicate a Design