Testing Complex Systems with Hard‑to‑Change Factors Using Covering Arrays (2026-US-PO-2953)

Testing is a critical component of complex systems engineering, yet test engineers are rarely able to evaluate every possible combination of system inputs. Growing system complexity, limited budgets, and tight schedules demand more strategic approaches to test design. The challenge becomes even greater when certain inputs or factors are expensive or time‑consuming to change, forcing engineers to carefully balance test coverage with practical constraints.

In this talk, we explore how covering arrays can be used to efficiently test complex systems while maintaining meaningful coverage of the input space. Covering arrays are a powerful combinatorial testing tool that enable engineers to achieve specified interaction coverage using far fewer test cases than exhaustive testing. We then extend this approach to address the presence of hard‑to‑change factors by introducing covering arrays with split‑plot structure, allowing test plans to reflect real‑world testing constraints. Using JMP, we demonstrate how these structured covering arrays can be constructed in practice.

Finally, we present a real application of this methodology for testing a JMP add‑in, showing how incorporating split‑plot covering arrays can significantly reduce testing time and effort while preserving confidence in system behavior.