Backward regression for Mixture DOE analysis with regular (non pro) JMP?

[#1]

I am working on a DOE training where regular JMP will be used so for DOE analysis Generalized Regression can’t be used. This is a pity for GenReg SVEM results for mixtures are excellent, great tool!

With regular JMP I tried stepwise and all possible models for mixtures and notice quite non reproducible results so – as show in JMP webinars -  I switched to simple Standard Least Squares backward analysis which gives quite reproducible results.

However backward regression is a method that I do not recommend for standard DOE analysis (I prefer all possible models) so how do I explain to trainees that with regular JMP backward analysis this is preferred for mixtures? Is there any other alternative, better way for analysing?

[#2]

Hi @frankderuyck ,

I think part of the answer is that Mixture experiments are really quite different from other experiments. Mixture factor effects are, by nature, correlated. Correlation of effects means more ambiguity, which means that it is more likely that you will find competing models that are similarly good fits to the data. Hence, a more conservative approach to model selection will be more useful.

I hope this helps,

Phil

[#3]

Hi Phil, this is also what I expected as answer, thanks!

One more question: as p-values are not indicating mixture effect significance is the Logworth pareto plot stil a good indicator for the strenght of an effect?OK that one can still eleminate low logworh effects? 

[#4]

Hi Frank,

If you choose Estimates>Cox Mixtures the p-values will be more recognizable and in agreement with the Profiler.

[#5 · Accepted solution]

Hi @frankderuyck,

As @Phil_Kay mentioned it, Mixture designs are quite different from other designs (like factorial designs), as the factors are not independent : changing the level of a mixture factor has an impact on others, so mixture factors are correlated.

Also, the emphasis of this type of design is more on predictivity and optimization than screening/statistical significance.
Last, you seem to have chosen a model-based mixture design (not a Space-Filling approach), so you already have assumed a possible complete model you would like to investigate.

For these 3 reasons, it makes more sense in the analysis to start from the full model with the possible terms you have assumed in the design creation, and start removing terms in the model (except main effects), based on the predictive performance of the model (RMSE for example), NOT on individual p-values/logworth of each term (because of multicollinearity/correlation among mixture factors, no intercept in this type of model, p-values/logworth are not a valid metric for model selection).

If you want to use a Stepwise approach to "automatize" the model selection, you can use Backward Stepwise Regression, with an information criterion (AICc or BIC, to balance accuracy vs. complexity of the model) as stopping rule. 

I hope this additional answer will help you,

[#6]

Hi Victor, thanks again for your useful input! I will try the Backward Stepwise selection tool with the AICc criterion. Is is good that in a basic trainng a standard analysis tool is used so I will use simple Stepwise as well for regular DOE however with foreward regression. I know that it is not perfect but to train & teach DOE power using clear cut cases this works still fine.

[#7 · Accepted solution]

As for mixtures prediction is the main goal, I assume that the DOE criterion is I-Optimal?

[#8 · Accepted solution]

I must admit I'm a bit confused by some of your posts in this thread, but here are my thoughts:

The primary analysis for mixture designs is the mixture response surface.  You should already know the model, you are exploring the surface to determine where you will get optimal results.  Think of it as a topographic map of the response. Visualizing where you want to "sample" that surface helps you think about the combinations of the factor settings. If you are running mixture designs (or any optimization designs), you should already know about factors and noise that affect consistency and predictability (these should have been studied in earlier screening designs).

[#9]

As p-values for mixtures are not indicating significance it will be hard to screen out significant effects; also, because of strong collinearity the precise mixture model is hard to find and is not the issue; specifying a good predictive and useful model is the goal so a pragmatic approach is required. I see in webinars that the logworth pareto with is used for backward selection of effective model parameters so I will use this in my training. Predicition here is the goal so is it not better to build an I-optimal DOE?