How to repeat points across extremities in a custom DoE to better understand variance?

[#1]

Hi!

I am currently optimising an experimental procedure and have opted for a custom design with 4 continuous variables with 2 levels each and a single 3 level-categorical variable. I have included centre points (replicated) to allow for some analysis of variance and the model is generated with RSM interactions not going beyond 2 factor interactions. The residual by plots prove to be very useful but only in relation to the output. Is there a way in which one could see whether there is an increase in variance (decrease in repeatability) in association with an extreme end of one of the factors/interactions. i.e. how can I assess variability in relation to the factors in the design space as opposed to just the response. 

Thank you for your help before hand!

[#2]

Hi Victor!

Thank you for your detailed response and  for the breakdown of variance sources, I found it incredibly helpful.

1- Model variance wasn't something I considered but is an excellent point! What's interesting when looking at this is that one of the recommended factors that the model suggested based on my desirability criteria falls into a range where the model variance is highest. I'm not sure if this should be a concern given that the ANOVA and lack of fit analyses of my model suggest that the model is well fitted and can use the input factors to predict the response.

2 and 3. This is exactly what I'm looking for - the only issue here is expense, ideally I'd like to repeat every experimental run but I simply will not have the resources to do so. Is there anyway to select certain points that I can model that will allow me to assess the response + input variance?

Thank you for your help and suggestions thus far!

[#3 · Accepted solution]

Hi @aaidaa,

Happy New Year ! And thank you for your response.

Looking at your different points :

1. Finding an optimum in an area of large(r) variance is an interesting situation. What you have to figure out is how large is this prediction variance compared to your target and expectations. It may be wiser to do some validation runs in this optimum settings, to decrease variance and assess the real predicted optimum performances/responses at this point.
   
   
2. (& 3.) As described in "Optimal Design of Experiments: a case study approach" by Bradley Jones and Peter Goos : "The best way to allocate a new experimental test is at the treatment combination with the highest prediction variance". In order to optimize your efforts, you can iteratively create new runs at locations with the highest variance in your experimental space.
   - Looking at the model variance, you can look at the script "Evaluate Design" and in the red triangle of the "Prediction Variance Profile", click on "Maximize Variance". This will give you the settings of the factors where the model variance is the highest, and can provide a good direction on where to add a new experiment in your DoE.
   - For the input and response variance, if you already have knowledge on the variance of the factors (and/or on the response measurements, thanks to previous MSA studies for example), you can also use this information through the Simulator (jmp.com) platform to be able to create simulated distributions of your responses at the optimum settings and evaluate mean and standard deviation of your different responses, given the variance of inputs and responses you have entered.
   - If you're looking at the "final/total" variance (which will probably be a mix of model variance, response variance and input variance if you have replicates), one way to continue could be to save the column "PredSE" of each of your responses, and using the profiler (from "Graph" menu, then "Profiler") with the formula of predicted standard errors of your responses (and then search to maximize PredSE of your responses, with the possibility to change the relative importance of your responses if it is relevant for your case) to determine where you can focus your efforts and repeat or create new experimental runs. You can also have a look at the Design Space Profiler (jmp.com) platform from JMP 17 to assess if you're able to find optimum points (and how much of the samples would be in specs), given some constraints/specifications on your responses target. You can also add PredSE of your responses to specify a constraint on the standard deviation of each of the responses if you have an idea on the precision you would like to have.

All these approachs are quite complementary, and can be really helpful to focus the efforts on the most informative experiments to run.

I hope these new comments will help you, 

[#4]

Happy new year! and thank you so much everyone this discussion has been super helpful!