Reinforcement Learning with Advanced Neural Network Architectures for Test Case Prioritization

Abstract

This thesis investigates the application of contemporary neural network architectures, specifically Convolutional Neural Networks (CNNs), Dueling Networks, and Transformers, to enhance Test Case Prioritization (TCP) within Continuous Integration (CI) environments using the RETECS framework and reinforcement learning techniques. Traditional TCP methods often fail to keep pace with the rapid development cycles and complex codebases characteristic of modern software development. By integrating these advanced deep learning techniques within the RETECS framework, this research aims to improve both the efficiency and effectiveness of the TCP process. A rigorous experimental setup, utilizing real-world datasets, was employed to train and evaluate the proposed models against traditional TCP methods. Performance was assessed using key metrics such as the Average Percentage of Faults Detected (APFD) and Normalized APFD (NAPFD), focusing on the models’ capability to efficiently detect faults and reduce testing time. Results indicate that the advanced models, especially the Dueling Networks, outperform traditional methods. Specifically, the Dueling Networks demonstrated a consistent increase in APFD scores by up to 3% over conventional models, showcasing their potential to optimize test case scheduling and fault detection within CI pipelines. These findings underscore the transformative potential of neural networks in automating and optimizing TCP processes within CI frameworks, suggesting a shift toward more intelligent testing systems. Moreover, the study highlights the critical role of tailored reward functions in reinforcement learning-based TCP to further enhance the prioritization efficacy of these models. This research contributes to the fields of software engineering and machine learning by providing empirical evidence of the benefits of applying sophisticated computational models to TCP. It also outlines future research directions, including the exploration of additional machine learning models and the integration of these methods into real-world CI environments to further refine and enhance the TCP process.