Easy DOE - Guiding New Users Through Designing and Analyzing an Experiment
tom_donnelly
| Updated on
Updated with new video April 2024.
Why use DOE and the Guided mode of Easy DOE?
- More rapidly answer “what if?” questions
- Identify important factors when faced with many
- Do sensitivity and trade-space analysis
- Optimize across multiple responses
- By running efficient subsets of all possible combinations, one can – for the same resources and constraints – solve bigger problems
- By running sequences of designs one can be as cost effective as possible and run no more trials than needed to get a useful answer
See how to:
- Start with the end by presenting DOE results interactively to decision makers
- Use the 6-step DOE Process implemented in the Easy DOE interface
- Understand factor types supported and model choices
- Use Guided Easy DOE process for slightly more complex 3-response, 4-factor, trade-space/optimization example using new .jmpdoe file.
- Define
- Specify
- Design
- Enter Data
- Analyze
- Predict
- Report
Questions answered by Tom Donnelly @tom_donnelly and Wendy Tseng @wendytseng at the live 2024 webinar:
Q: What if the factors are hard to change?
A: If you have hard-to-change factors, you need to use the Flexible mode in Easy DOE (vs. the Guided mode which is today's focus).
Q: How are the factors determined?
A: The experimenter is the one who determines which factors they want to study.
Q: How do you repeat the same results if they change each time? Is there a seed number we can control?
A: The simulator is simply a way to "run the DOE" since we don't have a lab. This allows us to design and analyze an experiment in this demonstration. (One could remove the random error in the simulator.)
Q: Why would you add a response without a goal?
A: There is an option to Add a Response without a goal (not a goal without a response) because you either do not have a goal for the factor yet or you simply do not have one (you just want to observe the response).
Q: What is the formula for the simulator columns?
A: s a formula that Tom built to make the experiment's results interesting. He made it up. Think of it as a way to collect data in a lab instantaneously for the purposes of this demo.
Q: Is there a table option to add p-values in the model estimate window?
A: You can right-click in the table and add the p-value. Right click > Columns > Prob > |t|
Q: What does it mean when JMP zeros the least significant factors?
A: The term is removed from the model. It does not play a role in changing the responses (i.e. speed, contrast)
Q: What does the scale refer to, in between term and the estimate column?
A: It represents the estimate for the model term. it is graphing the estimate values (estimate and the lower and upper 95%).
Q: What does the estimate indicate?
A: Model estimates are how the terms relate to the response.
Q: How can we reduce noise in DOE?
A: You can add more runs to your DOE.
Q: How does JMP create the sample data for modeling the relationship?
A: If you are asking about Simulate responses, JMP uses a normal distribution with mean of 0 a std dev of 1 as a default to add error to the model specified in the design with the coefficients you type in.
Q: If there is no replicates, how do you estimate error?
A: JMP's recommended number of runs includes more than the minimum number of runs to estimate the model (more runs than the # of model terms). Guided mode ensures that you have enough runs to get model error. (Replicates are necessary for pure error and testing lack of fit.)
Q: So, am I correct that one use for the Easy DOE is as a screening study?
A: Yes, you can use Easy DOE to design screening studies or optimization studies (or something in between).
Q: So essentially you need to run the experiments in to get data to simulate your model?
A: Yes, in real life, you are in Easy DOE to design the experiment. Then you go into the real world (e.g., lab) to collect your data. Then you type the data into Easy DOE (or load the data table) and do the analysis.
Q: Do you have an example with categorical x and y values?
A: You are seeing an example with categorical factors (X's). Categorical Y's are possible but a much more advanced topic (to ensure you are collecting enough data for a useful model).
Q: How does DOE result change if you switch one of the responses from minimize to maximize or vice versa?
A: the design is driven by the model choice. However, I can set the desirability, and for example change maximizing MOP to minimize MOP. This is going to change the answer, and then I reanalyze now that I've made that change when I optimize. In our case it the answers were pretty similar, but normally I would think if you were changing the goal there would be a difference. See video:
Q: I am using JMP16. Since there is no "easy DOE" I'm guessing I start with a "custom design"
A: Yes, you can use Custom Design in JMP 16.
Questions answered at previous weinars on this topic:
Q: if you are trying to detect a treatment effect on a process and have some prior knowledge on variation, is there a way within easy DOE to employ a power analysis for optimizing sample size?
A: Easy DOE does not have a power analysis within the platform but if you export the data on the data entry tab, there is a script in the resulting data table for design evaluation which includes power analysis. You can find various power analysis functions under the main DOE heading.
Q: Is there a way to visualize the design graphically?
A: Under the data entry tab, there are factor plot graphs that show each individual factor. The prediction profiler is also a way to visualize the effects of all the factors together.
Q: Is there information about what type of design was generated through this platform? For example, if it was an optimal design (and which optimality criterion was used), or if it was a fractional factorial (and if so what was the generator), etc. Basically, where can we find the "behind-the-scenes" info?
A: Here are the Easy DOE selection rules and how Easy DOE chooses each type of design.
Q: Is there information about what type of design was generated through this platform? For example, if it was an optimal design (and which optimality criterion was used), or if it was a fractional factorial (and if so what was the generator), etc. Basically, where can we find the "behind-the-scenes" info?
A: Easy DOE constructs designs using either the Custom or Definitive Screening Designs.
Main Effects: Custom Design (D-Optimal)
Main Effects (Uncorrelated with Two-Factor Interactions): Definitive Screening Design or Custom Design: (Alias Optimal)
Main Effects (Including all Two-Factor Interactions): Custom Design (D-Optimal)
Response Surface Design: Custom Design (I-Optimal)
Q: When using the profiler after fitting a multi-factor model, could you comment on what, if any, differences in modeling take place if you 'fit together' vs. 'fit separately'?
A: I'm interpreting your question as fitting the multiple responses in one profiler versus as 3 separate profilers. When you have all 3 responses in one profiler, you can look at the trade off in factor settings between the predicted responses of the 3 responses. The modeling is the same, but the optimization is different.
Q: In a manufacturing process, there may be multiple factors that go into assembly. There may be a factor that can be eliminated. How do you use DOE to identify this factor that can potentially be eliminated from the MFG assembly process?
A: When analyzing the data from a DOE, you would see the factor that can be eliminated will not be statistically significant as having an effect on the response. In the profiler that Tom showed, the effect would be a flat line versus having a slope
Q: How do you check for data reliability and/or does the easy DOE spit out Cronbach's alpha?
A: Easy DOE does not include Cronbach's alpha. That can be found in JMP's Multivariate platform using the Item Reliability option that each show or hide an item reliability report. The reports indicate how consistently a set of instruments measures an overall response, using either Cronbach’s α or Standardized ). See Item Reliability.
Q: Our study easily have more than 30 factors in the system. Our main goal for the DOE is trying to understand if there is any interations between this 30 factors. Which model is best to choose?
A: The second and third model options both will see if there are any interactions. It can be a difference in the number of runs needed and the trade off in how "expensive" in time or money each run costs.
Q: What is the difference between first order and second order models?
A: When we have a large number of factors, and we just want to find the critical few and investigate those critical few with more complexity, we would move from the first order model up to a second order model where we're trying to make predictions.
Q: Where is the simulator that you plug in the data
A: I added the simulator to the data table that I used as part of the demo. But there is also this simulation panel. If you know the coefficients, you can plug them in. So the short answer is, for the simulator built into JMP is a great for quick checks to find if this model will be able to be analyzed. If you're truly trying to simulate a process, you need to get your knowledge of the model fit and those coefficients into JMP.
Q: So by default, no replicates are added? Do we need to add them separately?
A: You can add replicates using the Flexible (more advanced) Mode.
Q: In many of the experiments that are ran by my team, many of the continuous factors are ran as discrete numeric factors. Are there any advantages and disadvantages for using this approach?
A: The only advantage would be, some people want more intermediate levels. If you're ultimately going to fit a quaradratic model, all you need are 3 levels. For example, to force 5 levels and you just pick a discrete numeric and give it 5 levels, JMP will probably put them in. Then JMP will pick way more of the extremes and fewer in the middle, because mathematically it's still thinking continuously with it. There is also more leverage at the extremes with categoricals. In summary that is a way to force more levels for a continuous, but if it's truly continuous, probably you don't need to do that.
Q: I use another DOE software that usually contain a set of lack-of-fit runs in a design. How was lack of fit tested in JMP DOE design?
A: Replicates will always give you a lack of fit test, because then we have an estimate of pure error that can be compared to the model error. If you don't have any replicates, we don't give you a lack of fit test but give you the same model that it will have some model error. If you've been making the same product for years, you probably already have a good historical idea of what the pure error is. One reason to do replicates nad tget the lack of fit test is if you are new to the process.
Resources
- JMP documentation on Easy DOE.
- Mark Baily's Developer Tutorial on Easy DOE
