Automated Alignment of Software Requirements and Test Cases

Publicerad

Typ

Examensarbete för masterexamen
Master's Thesis

Modellbyggare

Tidskriftstitel

ISSN

Volymtitel

Utgivare

Sammanfattning

Software testing can be expensive, but there are numerous tools available to enhance its efficiency. Nevertheless, certain areas still lack adequate technology. To address this, we conducted a qualitative study to identify shortcomings in existing tools and determine what features are needed in new ones. Through Thematic Analysis of the interviews conducted, we discovered that system-level testing stands out as an area in need of improvement, with a particular emphasis on the importance of well-defined requirements for effective testing. In light of this, we propose the development of a tool that assists testers in aligning their tests with requirements and generates suggestions for new tests using a powerful Large Language Model (LLM). The tool demonstrated great efficacy in terms of accuracy and recall, achieving an average of 86.394% each. It also provided valuable suggestions for testing approaches related to requirements. However, we observed limitations in its ability to achieve perfect alignment between requirements and test cases, as it showed a tendency for false positives. This led to the tool having an average precision of 45.582%. Moreover, the tool exhibited high efficiency when handling smaller input sizes. However, as the input size increased, we noticed a more than linear growth in analysis time, suggesting potential scalability challenges.

Beskrivning

Ämne/nyckelord

Software testing, Requirements Engineering, REST, Requirements and Test Alignment, Prompt Engineering, Large Language Models (LLM), Tool Development, Design Science, Computer Science and Engineering, Master Thesis

Citation

Arkitekt (konstruktör)

Geografisk plats

Byggnad (typ)

Byggår

Modelltyp

Skala

Teknik / material

Index

item.page.endorsement

item.page.review

item.page.supplemented

item.page.referenced