Using Design of Experiments Methods for Efficient Modeling & Simulation

By sequentially running High Performance Computer (HPC) simulations in Design of Experiments (DOE) blocks, one can quickly increase the accuracy of fast surrogate models. When sufficiently precise surrogates have been obtained, HPC resources can sooner be made available to other Modeling & Simulation (M&S) projects.  

See how to:

• Understand why to create a designed experiment before attempting modeling and simulation
• Compare and contract traditional DOE vs. Space-Filling DOE
• Decide which types of design to use – Latin Hypercube or Space-Filling
• Create a Space-Filling design, explore it visually, apply regression and machine learning models, create and honest assessment that includes training/test/validation data

NOTE: Q&A is interspersed throughout the video.

Resources:

• NASA Comparative Assessment Document
• VIDEO: C-5 Decision Support Tool Dashboard

• 2017 Dissertation by LTC Brian Wade, Director, TRAC-MTRY, Naval Postgraduate School 

• 2020 Military Operations Research Society (MORS) Modeling & Simulation (M&S) Community of Practice (CoP) Tutorial

• JMP Discovery Summit 2020 webcast of JMP BEAST Mode:  Boundary Exploration through Adaptive Sampling Techniques

• Design and Analysis of Computer Experiments  by Sacks, J., Welch, W.J., Mitchell, T.J. and Wynn, H.P.  Statistical Science 4. 409-423, 1989

• Santner, T. J., Williams, B. J., and Notz, W. I. (2003), The Design and Analysis of Computer Experiments, Springer, New York (2nd in 2018)

• Fang, K. T., Li, R. Z., and Sudjianto, A. (2005), Design and Modeling for Computer Experiments, Chapman & Hall/CRC Press, New York

• Kleijnen, J. P. C. (2008), DASE: design and analysis of simulation experiments. Springer, New York. (2nd in 2015)

• The Elements of Statistical Learning – Data Mining, Inference, and Prediction by Hastie, Tibshirani, and Friedman – 2001 (Chapter 7: Model Assessment and Selection)

• The Simulation Experiments & Efficient Design (SEED) Center for Data Farming at Naval Postgraduate School

• Naval Postgraduate School Designs https://nps.edu/web/seed/software-downloads

• Library of Orthogonal Arrays maintained by Neil J. A. Sloane

• Library of Orthogonal Arrays maintained by Warren F. Kuhfield