Choose Language Hide Translation Bar
Highlighted
Level I

## Analysis of Opt DOE simulation results

Dear JMPer,

Currently, I'm under analysis a Opt DOE simulation results. However, I don't get consistent result form simulations and also get to different trends during counter profile analysis. I wonder if any methodology or analysis could use to understand which simulation result is valid.

Here is my simulation process. The DOE matrix is in Table 1

1. Fit model (Simulation 1):
1. Put 4 inputs in Simulation 1 and select surface response mode in macro functions.
2. Remove interaction option from same factor, eg input A * input A
3. Run the simulation with standard least squares and effect leverage.
4. Table 2 shows Rsquare of summary of fit and Output trend based on Input A/C (Input A/C is key factors for the process)
2. Fit two level screening (Simulation 2):
1. Select Output A to run fit two level screening.
2. Use defined important factors, such as output's interaction and individual outputs.
3. Run the simulation with standard least squares and effect leverage.
4. Table 2 shows Rsquare of Summary Of Fit and Output trend based on Input A/C (Input A/C is key factors for the process)

Output B and Output C is able to predict the trend based on higher input or lower input (due to physic). However, Output A is the important response that I would like to see what simulation would show the trend. Based on Rsquare of Summary Of Fit, Simulation 1 seems batter than Simulation 2. However, output trend of Simulation 2 is more logical to me than Simulation 1. Therefore, I wonder if anyone could share some input to have better analysis for both simulation results. Please kindly share me your inputs. Much appreciate your time and help.

Thank you,

Bests,
Andy

Table 1. DOE matrix

 Leg# Pattern Input A Input B Input C Input D Input E Input F Output A Output B Output C 1 +−−++− 300 200 100 3 25 1 0.0% 2,463 61 2 −+−+−+ 200 400 100 3 15 5 0.0% 3,572 45 3 ++−−++ 300 400 100 1 25 5 0.0% 3,990 47 4 −−−−−− 200 200 100 1 15 1 0.0% 3,245 31 5 +−−−−+ 300 200 100 1 15 5 0.0% 3,017 34 6 −−−+++ 200 200 100 3 25 5 0.0% 3,842 76 7 −+−−+− 200 400 100 1 25 1 0.0% 3,431 48 8 ++−+−− 300 400 100 3 15 1 0.0% 2,409 51 9 000000 250 300 110 2 20 3 0.0% 3,336 66 10 000000 250 300 110 2 20 3 0.0% 2,564 66 11 −++++− 200 400 120 3 25 1 0.0% 3,289 62 12 −−+−++ 200 200 120 1 25 5 0.0% 2,698 53 13 +−++−+ 300 200 120 3 15 5 0.0% 3,017 47 14 +++−−− 300 400 120 1 15 1 0.0% 3,460 31 15 −++−−+ 200 400 120 1 15 5 2.8% 3,503 28 16 −−++−− 200 200 120 3 15 1 0.0% 2,835 42 17 +−+−+− 300 200 120 1 25 1 0.0% 2,304 42 18 ++++++ 300 400 120 3 25 5 0.0% 3,431 69 19 POR 150 300 120 0 0 0 7.1% 3,642 16 20 POR 150 300 120 0 0 0 8.3% 3,093 15

Table 2. Simulation 1 and Simulation 2 Results.

 JMP analysis Simulation 1 Simulation 1 Simulation 1 Simulation 2 Simulation 2 Simulation 2 Matrix Output A Output B Output C Output A Output B Output C Input A high Reduce reduce two trends increase Increase reduce Input A low increase increase two trends reduce Reduce Increase Input C high Increase Increase reduce reduce Increase Reduce Input C low reduce reduce increase increase reduce increase Summary of fitRsquare 0.96 0.78 0.84 0.95 0.59 0.85
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Staff

## Re: Analysis of Opt DOE simulation results

You asked, "Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening?"

First of all, this is NOT a simulation. It is a regression analysis to fit a linear model.

There are several common ways to approach the analysis of an experiment based on a fractional factorial design. You can approach it, as I said, using two stages in which you first select the main effects, believing that the principles of hierarchy holds, and then, add interaction terms for the active factors, believing that the principle of heredity holds. This approach is often a successful way to select the model. This model, like all others, must be verified with new empirical observations (augmented runs).

You asked, "Could you share me script for stepwise platform simulation?"

You do not need a script. Start your analysis with Fit Model as usual. Specify the response data column in the Y role. Specify the terms in the linear predictor for the Effects. But switch the Personality from Standard Least Squares to Stepwise..

Please see JMP > Books > Fitting Linear Models > Stepwise chapter.

Learn it once, use it forever!
5 REPLIES 5
Highlighted
Staff

## Re: Analysis of Opt DOE simulation results

So you designed an experiment for a computer simulation? That is, the factor levels are used as parameter values in the computer simulation?

I instead interpret your question to be about regression results. The two-level fractional factorial design in 16 runs with 2 center points and 2 ad hoc runs does not support estimating the full quadratic model. In this case, you must not try to estimate the parameters for the quadratic terms.

The Screening platform is intended only for two-level factors. You used a two-level design but you added a third or fourth level with the center points and ad hoc runs. This platform won't necessarily give you wrong information but it won't necessarily give you correct information when there are more than two levels.

Still, it can be useful as an exploration tool and a starting point leading to a full regression analysis. I used it to find a model for Output A with the linear predictor including Input E, Input D, Input A, and Input E*Input E. So it detected a non-linear response but attributing it to Input E is arbitrary. It is based on the heredity principle and nothing more. It cannot be established with this experiment. Output B does not appear to have any fixed effects from these factors. Output C has the most complex model. It includes two quadratic effects, which is the most that might be estimated and the choice of which terms is again arbitrary. The usual action at this stage is to augment the two-level design specifically to estimate all the quadratic terms for a final decision.

Another approach is to use the Stepwise platform starting with all the potential terms (main effects and interactions) that might be estimated. Here is the prediction profiler launched with the model formulas saved for Output A and Output C using stepwise regression to select the model.

Another approach is to start with the main effects only, remove the unimportant factors, then add and test interactions involving the important factors. This approach is based on the hierarchy and heredity principles.

Just remember that there are many confounded and correlated effects using this design. It provides incomplete information about the response. So augmentation might be the best next step to clarify the important effects. Regardless, the final model selected here or after augmentation must be empirically verified with new, independent runs for as yet untested treatments.

Learn it once, use it forever!
Highlighted
Level I

## Re: Analysis of Opt DOE simulation results

@markbailey  Thanks a lot for your detail explanation. I acknowledged inputs for the current matrix design.

The DOE matrix is designed for fractional factorial with some 2-factor interactions and I tried to define key factors for the process.

A quick question. Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening? I notice that you would analyze the defined effects and plot prediction profile.

Also, I haven't use stepwise platform before. Could you share me script for stepwise platform simulation?

Once again, thanks for your thoughts and inputs.

Thank you,

Bests,
Andy

Highlighted
Staff

## Re: Analysis of Opt DOE simulation results

You asked, "Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening?"

First of all, this is NOT a simulation. It is a regression analysis to fit a linear model.

There are several common ways to approach the analysis of an experiment based on a fractional factorial design. You can approach it, as I said, using two stages in which you first select the main effects, believing that the principles of hierarchy holds, and then, add interaction terms for the active factors, believing that the principle of heredity holds. This approach is often a successful way to select the model. This model, like all others, must be verified with new empirical observations (augmented runs).

You asked, "Could you share me script for stepwise platform simulation?"

You do not need a script. Start your analysis with Fit Model as usual. Specify the response data column in the Y role. Specify the terms in the linear predictor for the Effects. But switch the Personality from Standard Least Squares to Stepwise..

Please see JMP > Books > Fitting Linear Models > Stepwise chapter.

Learn it once, use it forever!
Highlighted
Level I

## Re: Analysis of Opt DOE simulation results

Thank you for correction. Again, appreciate for your suggestion.
Highlighted
Level VII

## Re: Analysis of Opt DOE simulation results

An alternate approach to Mark's method of model building (starting with 1st order then adding 2nd order based on excellent principles of scarcity, hierarchy and heredity) is to start with a saturated model.  Use Daniel plots (normal/half normal), Pareto plots (making sure the estimates are of practical significance) and Bayes plots (if your into that sort of thing).  Remove insignificant terms (consider Rsquare vs. Rsquare adjusted deltas), and re-run the model paying particular attention to residuals to ensure your model meets the usual assumptions.

One of my favorite quotes:

"Two equally competent investigators presented with the same problem would typically begin from different starting points, proceed by different routes, and yet could reach the same answer. What is sought is not uniformity but convergence" (Box, Hunter Hunter "Statistics for Experimenters").

Article Labels

There are no labels assigned to this post.