topic Re: Analysis of Opt DOE simulation results in Discussions
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247336#M48545
<P><LI-USER uid="5358"></LI-USER> Thanks a lot for your detail explanation. I acknowledged inputs for the current matrix design. </P><P><SPAN style="font-family: inherit;">The DOE matrix is designed for fractional factorial with some 2-factor interactions and I tried to define key factors for the process. </SPAN></P><P>A quick question. Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening? I notice that you would analyze the defined effects and plot prediction profile. </P><P> </P><P>Also, I haven't use stepwise platform before. Could you share me script for s<SPAN>tepwise platform simulation? </SPAN></P><P> </P><P><SPAN>Once again, thanks for your thoughts and inputs.</SPAN></P><P> </P><P><SPAN>Thank you, </SPAN></P><P> </P><P><SPAN>Bests,<BR />Andy </SPAN></P>Wed, 12 Feb 2020 20:20:17 GMTmandeson912020-02-12T20:20:17ZAnalysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247114#M48514
<P>Dear JMPer, </P><P> </P><P>Currently, I'm under analysis a Opt DOE simulation results. However, I don't get consistent result form simulations and also get to different trends during counter profile analysis. I wonder if any methodology or analysis could use to understand which simulation result is valid. </P><P> Here is my simulation process. The DOE matrix is in Table 1</P><OL><LI>Fit model (Simulation 1):<OL><LI><SPAN style="font-family: inherit;">Put 4 inputs in Simulation 1 and select surface response mode in macro functions.</SPAN></LI><LI>Remove interaction option from same factor, eg input A * input A</LI><LI>Run the simulation with standard least squares and effect leverage.</LI><LI>Table 2 shows Rsquare of summary of fit and Output trend based on Input A/C (Input A/C is key factors for the process)</LI></OL></LI><LI><SPAN style="font-family: inherit;">Fit two level screening (Simulation 2): </SPAN><BR /><OL><LI><SPAN style="font-family: inherit;">Select Output A to run fit two level screening.</SPAN></LI><LI><SPAN style="font-family: inherit;">Use defined important factors, such as output's interaction and individual outputs. </SPAN></LI><LI>Run the simulation with <SPAN style="font-family: inherit;">standard least squares and effect leverage. </SPAN></LI><LI><SPAN style="font-family: inherit;">Table 2 shows Rsquare of Summary Of Fit and Output trend based on Input A/C (Input A/C is key factors for the process)</SPAN></LI></OL></LI></OL><P>Output B and Output C is able to predict the trend based on higher input or lower input (due to physic). However, Output A is the important response that I would like to see what simulation would show the trend. Based on Rsquare of Summary Of Fit, Simulation 1 seems batter than Simulation 2. However, output trend of Simulation 2 is more logical to me than Simulation 1. Therefore, I wonder if anyone could share some input to have better analysis for both simulation results. Please kindly share me your inputs. Much appreciate your time and help. </P><P> </P><P>Thank you, </P><P> </P><P>Bests,<BR />Andy</P><P> </P><P> </P><P>Table 1. DOE matrix</P><TABLE><TBODY><TR><TD>Leg#</TD><TD>Pattern</TD><TD>Input A</TD><TD>Input B</TD><TD>Input C</TD><TD>Input D</TD><TD>Input E</TD><TD>Input F</TD><TD>Output A</TD><TD>Output B</TD><TD>Output C</TD></TR><TR><TD>1</TD><TD>+−−++−</TD><TD>300</TD><TD>200</TD><TD>100</TD><TD>3</TD><TD>25</TD><TD>1</TD><TD>0.0%</TD><TD> 2,463</TD><TD> 61</TD></TR><TR><TD>2</TD><TD>−+−+−+</TD><TD>200</TD><TD>400</TD><TD>100</TD><TD>3</TD><TD>15</TD><TD>5</TD><TD>0.0%</TD><TD> 3,572</TD><TD> 45</TD></TR><TR><TD>3</TD><TD>++−−++</TD><TD>300</TD><TD>400</TD><TD>100</TD><TD>1</TD><TD>25</TD><TD>5</TD><TD>0.0%</TD><TD> 3,990</TD><TD> 47</TD></TR><TR><TD>4</TD><TD>−−−−−−</TD><TD>200</TD><TD>200</TD><TD>100</TD><TD>1</TD><TD>15</TD><TD>1</TD><TD>0.0%</TD><TD> 3,245</TD><TD> 31</TD></TR><TR><TD>5</TD><TD>+−−−−+</TD><TD>300</TD><TD>200</TD><TD>100</TD><TD>1</TD><TD>15</TD><TD>5</TD><TD>0.0%</TD><TD> 3,017</TD><TD> 34</TD></TR><TR><TD>6</TD><TD>−−−+++</TD><TD>200</TD><TD>200</TD><TD>100</TD><TD>3</TD><TD>25</TD><TD>5</TD><TD>0.0%</TD><TD> 3,842</TD><TD> 76</TD></TR><TR><TD>7</TD><TD>−+−−+−</TD><TD>200</TD><TD>400</TD><TD>100</TD><TD>1</TD><TD>25</TD><TD>1</TD><TD>0.0%</TD><TD> 3,431</TD><TD> 48</TD></TR><TR><TD>8</TD><TD>++−+−−</TD><TD>300</TD><TD>400</TD><TD>100</TD><TD>3</TD><TD>15</TD><TD>1</TD><TD>0.0%</TD><TD> 2,409</TD><TD> 51</TD></TR><TR><TD>9</TD><TD>000000</TD><TD>250</TD><TD>300</TD><TD>110</TD><TD>2</TD><TD>20</TD><TD>3</TD><TD>0.0%</TD><TD> 3,336</TD><TD> 66</TD></TR><TR><TD>10</TD><TD>000000</TD><TD>250</TD><TD>300</TD><TD>110</TD><TD>2</TD><TD>20</TD><TD>3</TD><TD>0.0%</TD><TD> 2,564</TD><TD> 66</TD></TR><TR><TD>11</TD><TD>−++++−</TD><TD>200</TD><TD>400</TD><TD>120</TD><TD>3</TD><TD>25</TD><TD>1</TD><TD>0.0%</TD><TD> 3,289</TD><TD> 62</TD></TR><TR><TD>12</TD><TD>−−+−++</TD><TD>200</TD><TD>200</TD><TD>120</TD><TD>1</TD><TD>25</TD><TD>5</TD><TD>0.0%</TD><TD> 2,698</TD><TD> 53</TD></TR><TR><TD>13</TD><TD>+−++−+</TD><TD>300</TD><TD>200</TD><TD>120</TD><TD>3</TD><TD>15</TD><TD>5</TD><TD>0.0%</TD><TD> 3,017</TD><TD> 47</TD></TR><TR><TD>14</TD><TD>+++−−−</TD><TD>300</TD><TD>400</TD><TD>120</TD><TD>1</TD><TD>15</TD><TD>1</TD><TD>0.0%</TD><TD> 3,460</TD><TD> 31</TD></TR><TR><TD>15</TD><TD>−++−−+</TD><TD>200</TD><TD>400</TD><TD>120</TD><TD>1</TD><TD>15</TD><TD>5</TD><TD>2.8%</TD><TD> 3,503</TD><TD> 28</TD></TR><TR><TD>16</TD><TD>−−++−−</TD><TD>200</TD><TD>200</TD><TD>120</TD><TD>3</TD><TD>15</TD><TD>1</TD><TD>0.0%</TD><TD> 2,835</TD><TD> 42</TD></TR><TR><TD>17</TD><TD>+−+−+−</TD><TD>300</TD><TD>200</TD><TD>120</TD><TD>1</TD><TD>25</TD><TD>1</TD><TD>0.0%</TD><TD> 2,304</TD><TD> 42</TD></TR><TR><TD>18</TD><TD>++++++</TD><TD>300</TD><TD>400</TD><TD>120</TD><TD>3</TD><TD>25</TD><TD>5</TD><TD>0.0%</TD><TD> 3,431</TD><TD> 69</TD></TR><TR><TD>19</TD><TD>POR</TD><TD>150</TD><TD>300</TD><TD>120</TD><TD>0</TD><TD>0</TD><TD>0</TD><TD>7.1%</TD><TD> 3,642</TD><TD> 16</TD></TR><TR><TD>20</TD><TD>POR</TD><TD>150</TD><TD>300</TD><TD>120</TD><TD>0</TD><TD>0</TD><TD>0</TD><TD>8.3%</TD><TD> 3,093</TD><TD> 15</TD></TR></TBODY></TABLE><P> </P><P>Table 2. Simulation 1 and Simulation 2 Results. </P><TABLE><TBODY><TR><TD>JMP analysis</TD><TD>Simulation 1</TD><TD>Simulation 1</TD><TD>Simulation 1</TD><TD>Simulation 2</TD><TD>Simulation 2</TD><TD>Simulation 2</TD></TR><TR><TD>Matrix</TD><TD>Output A</TD><TD>Output B</TD><TD>Output C</TD><TD>Output A</TD><TD>Output B</TD><TD>Output C</TD></TR><TR><TD>Input A high</TD><TD>Reduce </TD><TD>reduce</TD><TD>two trends</TD><TD>increase</TD><TD>Increase</TD><TD>reduce </TD></TR><TR><TD>Input A low</TD><TD>increase</TD><TD>increase</TD><TD>two trends</TD><TD>reduce</TD><TD>Reduce</TD><TD>Increase</TD></TR><TR><TD>Input C high</TD><TD>Increase</TD><TD>Increase</TD><TD>reduce</TD><TD>reduce</TD><TD>Increase</TD><TD>Reduce</TD></TR><TR><TD>Input C low</TD><TD>reduce</TD><TD>reduce</TD><TD>increase</TD><TD>increase</TD><TD>reduce</TD><TD>increase</TD></TR><TR><TD>Summary of fit<BR />Rsquare</TD><TD>0.96</TD><TD>0.78</TD><TD>0.84</TD><TD>0.95</TD><TD>0.59</TD><TD>0.85</TD></TR></TBODY></TABLE>Wed, 12 Feb 2020 04:09:57 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247114#M48514mandeson912020-02-12T04:09:57ZRe: Analysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247219#M48526
<P>So you designed an experiment for a computer simulation? That is, the factor levels are used as parameter values in the computer simulation?</P>
<P> </P>
<P>I instead interpret your question to be about <EM>regression</EM> results. The two-level fractional factorial design in 16 runs with 2 center points and 2 <EM>ad hoc</EM> runs does not support estimating the full quadratic model. In this case, you must not try to estimate the parameters for the quadratic terms.</P>
<P> </P>
<P>The Screening platform is intended only for two-level factors. You used a two-level design but you added a third or fourth level with the center points and <EM>ad hoc</EM> runs. This platform won't necessarily give you wrong information but it won't necessarily give you correct information when there are more than two levels.</P>
<P> </P>
<P>Still, it can be useful as an exploration tool and a starting point leading to a full regression analysis. I used it to find a model for Output A with the linear predictor including Input E, Input D, Input A, and Input E*Input E. So it detected a non-linear response but attributing it to Input E is arbitrary. It is based on the <EM>heredity principle</EM> and nothing more. It cannot be established with this experiment. Output B does not appear to have any fixed effects from these factors. Output C has the most complex model. It includes two quadratic effects, which is the most that might be estimated and the choice of which terms is again arbitrary. The usual action at this stage is to augment the two-level design specifically to estimate all the quadratic terms for a final decision.</P>
<P> </P>
<P>Another approach is to use the Stepwise platform starting with all the potential terms (main effects and interactions) that might be estimated. Here is the prediction profiler launched with the model formulas saved for Output A and Output C using stepwise regression to select the model.</P>
<P> </P>
<P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Capture.JPG" style="width: 552px;"><img src="https://kvoqx44227.i.lithium.com/t5/image/serverpage/image-id/21716iAD4E538F770AD08A/image-size/large?v=1.0&px=999" title="Capture.JPG" alt="Capture.JPG" /></span></P>
<P> </P>
<P>Another approach is to start with the main effects only, remove the unimportant factors, then add and test interactions involving the important factors. This approach is based on the hierarchy and heredity principles.</P>
<P> </P>
<P>Just remember that there are many confounded and correlated effects using this design. It provides incomplete information about the response. So augmentation might be the best next step to clarify the important effects. Regardless, the final model selected here or after augmentation must be empirically verified with new, independent runs for as yet untested treatments.</P>Wed, 12 Feb 2020 13:51:32 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247219#M48526markbailey2020-02-12T13:51:32ZRe: Analysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247336#M48545
<P><LI-USER uid="5358"></LI-USER> Thanks a lot for your detail explanation. I acknowledged inputs for the current matrix design. </P><P><SPAN style="font-family: inherit;">The DOE matrix is designed for fractional factorial with some 2-factor interactions and I tried to define key factors for the process. </SPAN></P><P>A quick question. Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening? I notice that you would analyze the defined effects and plot prediction profile. </P><P> </P><P>Also, I haven't use stepwise platform before. Could you share me script for s<SPAN>tepwise platform simulation? </SPAN></P><P> </P><P><SPAN>Once again, thanks for your thoughts and inputs.</SPAN></P><P> </P><P><SPAN>Thank you, </SPAN></P><P> </P><P><SPAN>Bests,<BR />Andy </SPAN></P>Wed, 12 Feb 2020 20:20:17 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247336#M48545mandeson912020-02-12T20:20:17ZRe: Analysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247343#M48546
<P>You asked, "<SPAN>Based on your experience, should the simulation be focused on main effects which has defined by fit two level screening?</SPAN>"</P>
<P> </P>
<P>First of all, this is NOT a simulation. It is a regression analysis to fit a linear model.</P>
<P> </P>
<P>There are several common ways to approach the analysis of an experiment based on a fractional factorial design. You can approach it, as I said, using two stages in which you first select the main effects, believing that the principles of hierarchy holds, and then, add interaction terms for the active factors, believing that the principle of heredity holds. This approach is often a successful way to select the model. This model, like all others, must be verified with new empirical observations (augmented runs).</P>
<P> </P>
<P>You asked, "<SPAN>Could you share me script for s</SPAN><SPAN>tepwise platform simulation?</SPAN>"</P>
<P> </P>
<P>You do not need a script. Start your analysis with Fit Model as usual. Specify the response data column in the Y role. Specify the terms in the linear predictor for the Effects. But switch the Personality from Standard Least Squares to Stepwise..</P>
<P> </P>
<P>Please see JMP > Books > Fitting Linear Models > Stepwise chapter.</P>Wed, 12 Feb 2020 21:16:13 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247343#M48546markbailey2020-02-12T21:16:13ZRe: Analysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247344#M48547
Thank you for correction. Again, appreciate for your suggestion.Wed, 12 Feb 2020 21:21:15 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247344#M48547mandeson912020-02-12T21:21:15ZRe: Analysis of Opt DOE simulation results
https://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247642#M48609
<P>An alternate approach to Mark's method of model building (starting with 1st order then adding 2nd order based on excellent principles of scarcity, hierarchy and heredity) is to start with a saturated model. Use Daniel plots (normal/half normal), Pareto plots (making sure the estimates are of practical significance) and Bayes plots (if your into that sort of thing). Remove insignificant terms (consider Rsquare vs. Rsquare adjusted deltas), and re-run the model paying particular attention to residuals to ensure your model meets the usual assumptions. </P><P> </P><P>One of my favorite quotes:</P><P><I><SPAN>"Two equally competent investigators presented with the same problem would typically begin from different starting points, proceed by different routes, and yet could reach the same answer. <STRONG>What is sought is not uniformity but convergence" </STRONG>(Box, Hunter Hunter "Statistics for Experimenters").</SPAN></I></P>Fri, 14 Feb 2020 20:23:20 GMThttps://community.jmp.com/t5/Discussions/Analysis-of-Opt-DOE-simulation-results/m-p/247642#M48609statman2020-02-14T20:23:20Z