Doe adding data to the existed doe runs

[#1]

Hi,

i have designed and run custom doe. in addition i have results with the same parameters that i want also to put inside the fit analysis. is there any option to to that?

thanks 

[#2]

Here's what I suggest...create a second JMP data table that is the merged original design with the additional runs. How you do this can range from just hand entering the additional runs...or doing a data table concatenate. Then BEFORE analysis, I suggest running the Compare Designs platform to see how the bigger design compares to the original design. Here's a link to the Compare Designs platform.

Compare Designs 

Paying particular attention to the design diagnostics for estimability of effects. If the added points materially change in a negative way, your ability to estimate effects, you'll have to make a decision regarding the desirability of such a merger if effect estimability is very important to you. One very desirable characteristic of the JMP Custom Design platform is the design is model driven...hence the design will optimally support estimating the effects you articulate. By adding additional treatment combinations to the experiment you may inadvertently negatively impact the ability to estimate effects.

[#3]

Peter, hi! Thank you for a nice explanation. My colleagues also want to add some extra runs at some specific points in the design space. I don't understand, why would you do it? What is the rational behind this approach? Is there any specific term used for that (would help me to find some references)? Thank you and greetings.

[#4]

Hi @Aziza,

@P_Bartell suggests to run in parallel the two designs (original design and the design with extra points), to verify and calculate that the parameter estimates found in the original model are similar enough to the ones from the design with extra points, and/or that you don't miss any other terms/effects, and/or that you don't have a lack-of-fit with possible extra degree of freedom added with the new points.

Note that depending on the new values added (and the noise), it may change the terms in the model. You may need to :

• add new terms in your second model (if there was not enough points to estimate/detect them previously in the DoE or they were not significant enough),
• remove existing terms in your second model (because new points show an inverse/opposite behaviour compared to previous experiments done in the DOE, so effects are not significant anymore), 
• simply keep the same terms as before, with a possible increase in parameter estimation accuracy (depending on the experimental variance).   

One "other" option (to do perhaps before the models comparison) could also be to use the newly added points (not from the original design) as validation points (in the tradition of validation set in Machine Learning). This way, you check that the model created based on the DoE is accurate enough and may generalize to new experiments done by visualizing residuals and comparing your performance metric(s) (R², R² adjusted, RMSE, ... depending on your objective) between points in the model (training set) and newly added points (validation set).

Depending on the differences you find between results from the model on training and validation sets, you may have to explore the models more precisely and go to models comparison method, to discover and understand what are the differences and consequences of adding the new points.  

Hope this helps you

[#5]

I'll provide some thoughts on your questions:

1. "What is the rational (sic) behind this approach?" I think you meant "rationale"? So here's my thoughts. Let's start from the standpoint of adding runs BEFORE conducting the experiment. Someone might rationalize this by saying something like, well the optimal design does NOT include treatment combinations for which we most definitely require empirical results, hence let's just add them to the design rather than running a second experiment with these combinations. A matter of convenience and practicality mostly. Another example of this sort of thing was adding what we called 'checks' to the original experiment. A 'check' for us was a treatment combination(s) where we had a very good idea of what the responses would be and we were using the 'check' as a means to reassure ourselves there wasn't some big noise or nuisance factors influencing the balance of the results. We never included the 'check's' in the analysis. Big caveat is, the optimal DOE approach is model driven, and adding willy nilly combinations AND including those combinations in the analysis may inadvertently make it more problematic to get the information from the analysis you were seeking with the original design.

2. If adding the treatment combinations to the original design AFTER the original experiment has been conducted that's what many call, 'augmenting' the original design. The rationale behind this is often pretty simple...the original design's analysis did not lead to us being able to solve the practical problem at hand...but we think we are on the right track...and adding well thought out (using optimal DOE strategies and tactics) runs to the original design will lead us closer to the resolution of the problem at hand. Examples might be...we need to estimate interactions or quadratic effects that were not estimable from our original design. This is pretty easy to accomplish in JMP using the Augment Design platform. Here's a link to that section of the JMP online documentation that describes augment design strategies and tactics and 'how to' in JMP, including an example: Augment Designs in JMP 

Lastly for a reference and example, I suggest chapter 3 of Joos and Jones "Optimal Design of Experiments A Case Study Approach".

Hope this helps?

[#6]

Thank you so much, Peter for a very insightful and clear formulation of the answer. I understand better now. I will also make a good use of the references you mentioned. I appreciate your support. Sincerely.

[#7]

I will add my thoughts to the excellent suggestions already noted:

Adding additional replicates of the same treatment combinations "later" adds additional noise to the study. You should be careful about how you analyze the additional runs.  Since it appears that you (and/or your colleagues) are "selectively" choosing which treatments to replicate, be aware of this bias. (see papers on Randomized Complete (and incomplete, BIB) Block Designs), for example:

Sanders, D., Leitnaker M., and McLean R. (2002) “Randomized Complete Block Designs in Industrial Studies” Quality Engineering, Vol. 14, Issue 1

It can be hugely advantageous to increase the inference space by running replicates over changing noise.  If this is done systematically/purposefully, you might get significantly more insight to the mechanisms at work (which factors have an effect) and if those factor effects are robust to changing conditions (noise).  If this is done randomly, you increase the inference space, but negatively effect the precision of the design.

[#8]

Thank you so much for your input. I will take it into account and review the references. I appreciate your time and support.