Authors
Yi Ji (1), Simon Mak (2), Ryan Lekivetz (1) and Joseph Morgan (1)
Affiliations
(1) JMP Statistical Discovery LLC
(2) Duke University, Durham, NC
Journal
2024 22nd ACM-IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE)
Date Published
2024
Abstract
Software testing is essential for the reliable and robust development of complex software systems. This is particularly critical for cyber-physical systems (CPS), which require rigorous testing prior to deployment. The complexity of these systems limits the use of formal verification methods. Furthermore, testing and fault localization can be very costly. To mitigate this cost, we outline in this work a holistic machine-learning-guided test case design and fault localization (MaLT) framework, which leverages recent probabilistic machine learning methods to accelerate the testing of complex software systems. MaLT consists of three steps: (i) the construction of a suite of test cases using a covering array for initial testing, (ii) the investigation of posterior root cause probabilities via a Bayesian fault localization procedure, then (iii) the use of such Bayesian analysis to guide selection of subsequent test cases via active learning. The proposed MaLT framework can thus facilitate efficient identification and subsequent diagnosis of software faults with limited test runs. This framework has potential for integration with an assertion-based test oracle approach, which may prove to be an efficient and cost-effective way of integrating light-weight formal methods with testing.
Citation
Ji, Y., Mak, S., Lekivetz, R. and Morgan, J., 2024, October. MaLT: Machine-Learning-Guided Test Case Design and Fault Localization of Complex Software Systems. In 2024 22nd ACM-IEEE International Symposium on Formal Methods and Models for System Design (MEMOCODE) (pp. 58-62). IEEE, https://doi.org/10.1109/MEMOCODE63347.2024.00011.