Design of Experiments Club

Hosted by @HadleyMyers , DOE enthusiasts shared discusions about several topics.  Feel free to continue the discussions below!    Video 1: Introduction   Video 2: Dealing with outliers   Comments:  @Victor_G : And also separate repetitions from replications to better understand if the problem comes from measurement process (repetitions), or from the system (with possible uncertainty in fixing factors levels, seen through replications)   Video 3: Uses of covariate factors   Comments:    @Victor_G : Jed Campbell from JMP did a great blog post about non-linear factors constraints : https://community.jmp.com/t5/JMP-Blog/Demystifying-Factor-Constraints/ba-p/619898 Demystifying Factor Constraints Isn't Define Factor Constraints something I should just leave alone or pretend doesn't exist? A wise man does nothing by constraint. -Marcus Tullius Cicero If you've felt intimidated by the Define...   @Paul_J : Did that all the time when I worked in food and chem industries.   @Bill_Worley : I'm a big proponent of using covariate candidate sets.   @Victor_G : Covariate sets are very useful in chemistry, when dealing with chemical properties as inputs for example   Video 4: best practices on setting ranges   Comments:  @Victor_G : Use the broad range in screening (possibly with centre points), and refine the range in exploration/optimization.   @KimRas2 : Be bold!     @shs: What works for me, if you start from scratch and you are unsure if your borders work, start with the experiment which you think might be the most critical. Here you waste only one experiment.  @Victor_G : Scoping designs are also very useful to define appropriate ranges and check variability in design space : https://prismtc.co.uk/resources/blogs-and-articles/scoping-designs You only try tests with -1 levels for all factors, centre point(s) and +1 levels for all factors. And you can very easily augment your scoping design into a factorial design   @Nicola_Nuti: Agreed, increasing effect size gives more power as well.    @Jonas_Rinne : The idea of scoping designs is super helpfull to check if your ranges make sense. Especially when you have many factors.   @SDF1: Even trials that don't work still provide very useful information for the DOE and the whole exploratory process.   @shs : With too narrow settings, you can never be sure about curvature., which is often the case with temperature.   @Paul_J : setting the ranges is good example of why to keep the science in the stats.   @Bill_Worley : If you have some historical data you can build a model and use the Design Space Profiler to help determine optimal ranges.   @Victor_G : Yes, DoE is a method change, but with more predictability about the outcomes, learnings and with planning made a lot easier (compared to trying One Factor At A Time and relying on best guess).   @SDF1 : Depending on how "new" the research is, sometimes a "pre-DOE" is necessary to determine what are the workable ranges for the factors that you're testing so that when you actually do sit down to create a more detailed/complex DOE, you're more likely to have all runs be successful.   Video 5: Modern DOE´s   @Victor_G : Really like the metaphor with the roof, thanks for sharing!   @shs : I rarely use classical design anymore, as they are large in experimental runs. Using concepts like SVEM with small designs, or definite screening designs, etc.. can give you designs which can cover large amount of factors with few runs, and that convinces people when they first see how much the experimental efforts can drop using modern designs. @Victor_G :  I often evaluate and compare modern designs with a possible "classical" counterpart, to better show the benefit of the modern design   @KimRas2 : Functional DoE?   @ryan_cooper : I often like to refer to some of Phil Kay's points in this Community discussion.  (The title of the question is misleading, however.)  Some discussion on why most things can be done with DSD and Custom design. https://community.jmp.com/t5/Discussions/Reason-against-using-Custom-Design-DOE-for-everything/td-p/407995   Video 6: Functional Data Explorer   @Victor_G : Functional DoE with curve output : https://community.jmp.com/t5/Discovery-Summit-Europe-2021/Use-of-Functional-Data-Explorer-in-a-Mixture-Design-for/ta-p/349208 You can use curve data (functional data) as inputs and/or output.   Lyndal asked: Does the standard JMP license cover functional DoE analysis? I thought that needed JMP Pro? Yes, it´s only in Pro   @Victor_G : As an input, you can use coefficients from functional analysis, and link these coefficients to your outputs   @Phil_Kay : @Ben_Ingham has some expertise in that field. Ben will be at Discovery Summit in Manchester if you able to make it.           ... View more

Thanks @J_Asscher and @Phil_Kay for this session, and to all Users who participated!    Video:    Resoults from the Poll Questions:  The discussion continued though the chat:  Q: Do you always accept the default number of whole plots?  Would you ever increase it? A1: Very useful indeed, but sometimes a bit more problematic for modelling A2: You may have to increase the number of whole plot if you want more power and more precision in the estimates of your main effects for factors that are hard to change   Q: What is the implication of running a standard design by grouping experiments? is it ever acceptable? A1: You may have lurking variables influencing your responses, like time, temperature, operator, etc... and the error is not random anymore (it may correspond to changes due to the "block changes"), which may hide some important factors or reveal non-important ones correlated to lurking variables A2: imagine if you make two blocks and in the first you have X1 as -1 and on the second part as +1. If you have an hidden effect ore something degrade over time that increase over time, this effect is attributed to X1 even if its not. This can lead to false models.   Some further information was shared during the session:  - White Paper on Split Plot: https://www.stat.purdue.edu/~kuczek/stat514/Split%20plot%20example.pdf - Follow Phil on LinkedIn: #DOEbyPhilKay - Follow Victor Guiller on LinkedIn: https://www.linkedin.com/in/victorguiller/   ... View more

Thanks to @Victor_G and @ChristianBille for this first session!    Recording 1:    Poll 1: Which design are you most familiar with?   Poll 2: If you are not using optimal design, why not?   Some comments to continue the discussion online:    From @MartinNovak: I would love to hear more about gradually progressed optimal experiments/designs -> how to properly combine several subsequent groups/blocks of experiments statistically?    From Tin Huu Bui: How does JMP choose what point to test in optimal design? I'm research about a topic called Bayesian Optimization. That from each test, you will know what should you test next to gain biggest knowledge/ information. Can we consider it as a replacement for DOE? Answer: It depends on the optimality criterion and variance in your design.  You can try to augment a design based on real data or simulation data, JMP will choose points where the prediction variance is highest.   From @ruskicar: Do space filling designs in general require more experiments than model-based designs? Answer: Yes in general. If you do a comparative test in JMP, you'll see that optimal designs are more cost-efficient than space-filling designs. But they may not be used for the same problem,   Discussion after the Breakouts:    Comments from the discussion:  @martindemel : we tried DOE in the past and it did not work (quote often means they did not do a statisticially designed experiment when sking for more information what they did)  @Georg: too much noise -  Low power run or bigger test or reduce the expectation @Victor_G : Yes, interactions are often underestimated @ChristianBille : the cost is apparent upfront   The next Session will be on September 26th: https://www.jmp.com/en_gb/events/users-groups/users-group-meetings/doe-community-of-practice.html Please suggest topics that you would like to see covered in future sessions: https://community.jmp.com/t5/Design-of-Experiments-Topic/idb-p/doe-grouphubidea-board?_ga=2.214469079.1765168125.1687762975-2026663903.1684154484       ... View more

What is most important in the define stage of the statistical design and analysis of an experiment? Are there examples of experiments that have gone badly because something important was missed in the define stage?     ... View more