Custom DoE screening design

[#1]

I have two very basic questions about custom DoE screening design.

1. If I only wanted to know which main effects are significant to my responses and use the custom main effect screening design with the default run numbers given by JMP, how can I be sure whether it is the main effect or other confounding two-way interaction that is really significant? If the model analysis shows factor A is significant but A is partially correlated with B*C, what factor(s) should be included in the optimization design later? I suppose only the main effects that are significant such as factor A should be further investigated for the optimization design following the main effect screening design but will B and C be missing if B*C partially correlated with factor A but B and C are not significant main effects from the screening DoE?

2. when I include some uncontrolled factors in the model, JMP does not show the color map for correlation any more. How can I evaluate the design with uncontrolled factors?

Thanks

LL

[#2]

@statman is guiding you very well. My only additional comment from 'the sidelines' is that you are thinking in terms of old design methods. The design principles never change but the methods continue to improve. Custom design is a new method. It permits strategies that were impossible just two decades ago. But with the new capabilities comes the responsibility to learn both the principles and the ways of the new method.

For example, uncertainty in the active effects is an issue from the start of statistical DOE a century ago. After regular fractional factorial designs appeared for the sake of economy in screening experiments, the method dictated the strategy. It resulted in confounded estimates (perfectly correlated estimates).

Play close attention to @statman's questions about variance or noise. These are key questions that a greedy experimenter must address in order to get the most information with the most confidence from the smallest design.

[#3]

Follow up with the Custom DoE for screening main effects, below is the model fit results. Is it safe to say the factors with t-ratio smaller than 2 can be eliminated and only those factors with t ratios larger than 2 will be used for the optimization design using a RSM next? If p value is available for each factor, I would think the factors with p value smaller than 0.05 should be included in the RSM optimization DoE next. Any suggestions about this thought process will be appreciated. Thanks! 

[#4]

@JMP38401 sorry for the delay...It is difficult to look only at the output you shared and draw conclusions.  It looks like you have an un-replicated design and your model is saturated.  If this is the case, you might want to try getting Normal Plots for statistical significance (sometimes you have to ignore Length's PSE line), Pareto Plots for practical significance and Bayes Plots if you're into Bayesian philosophy (Fit Model>Red Option Triangle next to response>Effect Screening).  The first thing I always do is check for practical significance.  Did you create variation of any practical value?  What is the smallest increment of change in the response that you think is of scientific or engineering value?  After you establish this value, plot the line on the Pareto plot and identify which factors had a practically significant effect.  Then look for these effects on the Normal plot (Daniels Plot).

As far as optimization goes, it is extremely difficult to provide advise with the amount of context you've given.  Please realize optimization is far from just a statistical design.  It requires interpretation from someone who understands the science/engineering.  A couple of thoughts though:

1. You are not trying to create some incredibly complex non-linear model that describes everything.  Models are meant to be efficient approximations that are useful for prediction.

2. You should NOT be doing optimization of design factors unless you thoroughly understand noise.

3.  What did you mean by RSM?  G.E.P. Box implies this is sequential experimentation.  It is not one central composite design.

4. You also should be thinking multivariate.  Doesn't do any good to optimize one Y at the sacrifice of others.

[#5]

Thank you for the suggestions! I tried to get the Normal plot but all three plot options under Effect Screening were grayed out and not available to show.

"The first thing I always do is check for practical significance.  Did you create variation of any practical value?  What is the smallest increment of change in the response that you think is of scientific or engineering value?  After you establish this value, plot the line on the Pareto plot and identify which factors had a practically significant effect.  Then look for these effects on the Normal plot (Daniels Plot)."

I can estimate the standard deviations for all the factors and know the practically meaningful minimum shift in response but not sure about your suggestion, "After you establish this value, plot the line on the Pareto plot and identify which factors had a practically significant effect.  Then look for these effects on the Normal plot (Daniels Plot)." Can you please elaborate a little bit more on how to do it and why to do it? Thanks!

[#6]

I guess any factors with an estimated effect larger than the practically meaningful minimum shift on the Pareto chart are significant and they should deviate from the normal line on the Normal plot as well. Should both plots give the same conclusion? 

[#7]

No, you can have effects show as significant on the Normal plot and yet not have any practical significance.

[#8]

Again,  I can't really help without the data table.

[#9]

Questions about blocking for this design...

Since I will need multiple days to complete all the runs, I would like to block the design. However, since I have HTC factors in my design, I don't see the option for "group runs into random blocks of size". I can add a blocking factor into the factor list but I am not sure whether it is the right way to do it since the blocking factor (day) will be treated as fixed blocking.

[#10]

Are you interested in Balanced Incomplete Blocks or Randomized Complete Blocks?  I usually don't use the built in JMP blocking as it treats blocks as random effects.  I prefer to design experiments where I know what noise is confounded with the block and I assign the Block and Block-by-factor interaction effects to the model.  To do this, I copy one entire block, paste it to the first open cell in the first column under the first block, then I add a column and call it block (first block -1, second block 1).  I also typically am designing experiments with only 2 blocks (primarily industrial experiment applications vs. agricultural).  I am not interested in curvature (non-linear) of a block effect since this is non-sensical.

[#11]

Sorry...I am not able to follow the comments but all I planned to do is to separate the total number of runs into two blocks using day as the block factor. Is there a way to do it using JMP with HTC factors? Thanks!