What is a good r2 in Mixture Design DOE

[#1]

Coming from a field, where I only had contact with calibration curves, where only 0,999 is a good fit, I wonder what r2 would be considered good in Mixture dOE Modeling? Are there other values that are important to check if I want to know if I have a good model?

[#2]

IMHO, mixture designs are primarily optimization designs. Meaning, you have already done screening, have a reasonable first order plus model, understand noise and multivariate considerations. You are now at a point where you are selecting the sweet spot to run. This is primarily done via response surface plots (mixture response). The typical model building statistics can be challenging. There can be a fair amount of multicollinearity which can be completely acceptable, but makes traditional statistics and coefficients difficult to interpret.

With this in mind, Cornell seems to suggest an R-sq Adjusted minimum to be .85 (R-sq by itself is seldom useful), but if that is not meant, it just means an a modification to the model form may be useful.

See Piepel, Gregory, Cornell, John (1994) "Mixture Experiment Approaches: Examples, Discussion, and Recommendations", Journal of Quality Technology, Vol. 26, No. 3, July

The three basic steps of Mixture designs according to Snee (based on Box and Wilson Response Surface methodology):

1. Data are generated using experimental design

2. A model (usually polynomial) is fit to the data

3. The response surface contours are examined to determine the regions where the best values of responses can be obtained.

See Snee, Ronald (1971) "Design and Analysis of Mixture Experiments", Journal of Quality Technology", Vol. 3, No. 4, October

[#3]

my model is quadratic mixture model with R2 being 0.92 and adj R2 0.90 I just wondered, when do I reach the point where I can confidentially say: "my model is good enough", if that depends on R2 or more on how accurate my predictions are.. Testing a point which is not part of the model showed some deviation of the predicted one, which is to be expected, but for me it is hard to tell if it is just experimental variance or due to a bad model. Lack of fit is at 0.9

[#4]

There is no absolute statistical assessment of whether "my model is good enough". Good enough is not operationally defined. The best models are ones that make sense, from a subject matter point of view, and are useful from an operational standpoint.

[#5]

That Sounds good, but somehow there Must be Like a threshold where people would say: thats Definitely Not a good Model or fit. For my eyes as a user I am happy with the Model I have and what it Serves being Aware of it limitations but as I am new to the Field, I dont see myself in the Position to Tell of it is a good Executed DOE/Model

[#6 · Accepted solution]

There are plenty of clues that would lead you to conclude the model is not adequate. But again, you do not define good? Why would you think someone with great statistical knowledge, but no domain knowledge would be a better judge of the model adequacy? You want your model to predict responses such that the errors are randomly distributed around 0. The model needs to be consistent over changing conditions.

"A good model is an approximation, preferably easy to use, that captures the essential features of the studied phenomenon and produces procedures that are robust to likely deviations from assumptions"

                                                                                               G.E.P. Box

[#7]

I agree with everything @statman has contributed so far. One thing I'll add is I hope you are not looking just at R2 to evaluate the 'goodness' (and there is no such thing) of the model. If you are focused solely on R2 you are falling victim to the dreaded disease called mononumerosis. Instead answer this question: "Does the model adequately address the goals and objectives of the experiment at hand?" Quite frankly if you answer 'yes' to that question I don't care what value R2 has. When I worked in industry we were paid to solve problems...not get 'good' (yikes there I go again) statistics R2, p value, F ratio, mean square error, t statistics, and any other statistic you can think of.

[#8]

Thanks to Both of you you for the clear statements. The reason I mostly Looked into R2 so far, was my Prof, also new to DOE, exactly did that, probably because it is the only value he has seen before (in other settings, where it is an important value). Being also new to the theory of DOE and more using it to help me reaching the next research step, I Couldnt convinve him, that a working (Not Perfect) Model is more than sufficient. Next time I will Focus more on the practical benefits and outcomes than on statistical values. Thank you for the Input!