Investigation of approaches for change impact analysis and facilitating finding reviewers in testing frameworks

Abstract

As software becomes larger and more complex and time between deliveries decreases the more maintenance becomes an issue. An established way to facilitate some maintenance efforts is to use automated testing frameworks to reduce the time needed for testing, freeing up both computational and human resources for other tasks. However, as the number of testers using such a framework increase so does the maintenance efforts on the framework itself and there seems to be a gap in research regarding processes for improving maintainability in testing frameworks. This study aims to provide means of improving maintainability of testing frameworks by first investigating issues that hinder maintenance efforts in a real testing framework and search in literature for possible solutions that can be applied to address the issues. We implement a tool for change impact analysis to assist maintainers in making informed decisions during maintenance and we create guidelines for what to consider when selecting tools for automatic reviewer recommendation. We evaluate the tool and guidelines by interviewing experienced practitioners and the results show that maintainability of a testing framework can be improved by using tools for CIA and the guidelines provide a good starting point for practitioners wishing to learn about reviewer recommendation tools. Specifically, we found that by using our tool maintainers can more easily identify the impact of their modifications and understand the related components of the framework, allowing them to mitigate the risk of introducing faults by taking corrective actions. The interviewees pointed out that there are limitations of the tool in the current context, most importantly that a specific type of important files are not covered and that the impact set may sometimes be too large. These are limitations that should be addressed in future research as all interviewees agreed that if they could be addressed, the tool would be very valuable. Ultimately, our contributions to research are i) fostering transfer of research in CIA to industry and assessing its effect, ii) identifying challenges in implementing tools for CIA in practise and iii) providing means of understanding tools for reviewer recommendation. Thus, the end goal of the thesis goes beyond the deliverables we produce, additionally, we aim to generalize the results of the thesis such that they can be applied in other contexts where organizations using testing frameworks face similar problems. In the long term, our research lays the foundation for continuing transferring research to industry and helping researchers identify where their efforts are most needed in the industry.