A Case Study on Test Optimisation and Visualisation of Diversity Information

Abstract

Continuous Integration (CI) pipelines are vital in the implementation of CI and in the feedback cycles that surround automated testing in CI environments. A feedback cycle represents the time it takes from performing a commit until test results are ready. A significant problem in CI and automated testing is the long feedback cycles that come due to the increasing size of the test repository when executing test suites. The increased length of test execution is what this thesis will address using test case prioritisation. Through a design science methodology, we developed a tool and evaluated it by performing a case study at Volvo Car Corporation. The case study consists of two parts: evaluating the visualisation of data usually hidden during prioritisation, and data gathering and statistical analysis related to the performance of different distance measures and test case data. We have identified that similarity maps and history plots are good visualisation to enhance test decision making and maintaining and improving test repositories. Moreover, we have discovered the potential of using previous executions of test cases to determine their similarity.