Bayesian modeling approach to Test Case Prioritization

Abstract

Regression testing taking up valuable time and computational resources is a prevalent issue in software engineering. Lowering the time consumed by the execution of a regression suite enables timely feedback to developers and faster software verification. One technique to reduce the cost of regression testing is called test case prioritization. This project utilized an action research methodology to investigate a case company’s requirements specific to test case prioritization and evaluated a novel Bayesian modeling approach against a Heuristic approach. In addition, the results were discussed in relation to Machine Learning-based approaches. The different approaches were evaluated based on several elicited requirements on one open-source dataset and the case company dataset. The results showed that Bayesian modeling performed similarly to heuristic models and machine learning models in terms of early fault detection, even with limited amounts of training data. Furthermore, Bayesian models showed a higher average percentage fault detection than heuristic models on the open-source dataset. From the perspective of small to medium-sized companies (SMEs), common test case prioritization techniques may improve early fault detection, however, additional work may be needed to meet the requirements and demands of the companies’ testing and verification practices. A pragmatic approach to test case prioritization for SMEs could use a combination of Bayesian modeling and rule-based or heuristic prioritization.