Automated Requirements Review using Artificial Intelligence
Ladda ner
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Software engineering and technology (MPSOF), MSc
Publicerad
2023
Författare
Chand, Sivajeet
Li, Chang
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Amidst the perpetually shifting landscape of automotive software development, this study sets out on a journey to outline the desirable properties that define an ideal requirement and subsequently harness these attributes to propel the semi-automation of the requirement review process. By utilizing these properties, the study endeavors to offer insightful feedback aimed at enhancing the quality of individual requirements. The heart of this endeavor lies in exploring machine learning to effectively automate the requirement review process.
This project has been performed in collaboration with an automotive company, APTIV. Several software companies, APTIV included, have embraced a requirement writing model that employs natural language as the preferred approach for articulating software requirements. The intrinsic nature of these requirements—often composed in natural language by various authors—renders them susceptible to errors, inconsistencies, and ambiguities. To mitigate these challenges, manual validation by developers and product owners ensues. However, this manual approach incurs escalated costs, resources, and exertion. It is further compounded by the potential for misinterpretation, leading to undesired software attributes. Thus, the quest for an innovative solution, one marked by consistency and automated review of handwritten requirements, becomes paramount.
This thesis endeavors to automate the requirement review process within the automotive industry by applying machine learning. Through the lens of automation, we delve into the complexities of requirement review. To gain insights into the manual requirement review practices of automotive experts, a series of interviews were conducted with nine professionals. These interviews focused on understanding the attributes they associate with requirements, their existing review strategies, and challenges stemming from poorly formulated requirements. The information gathered from these interviews suggests that unambiguity, consistency, and verifiability are the three most important requirement properties in the automotive industry and also laid the foundation for the subsequent model training.
Five distinct amalgamations of language models and classification techniques have been meticulously trained to predict whether requirements meet these three properties. Classical models—Support Vector Machine (SVM), Naive-Bayes, and Random Forest—converge with the CountVectorizer method. Furthermore, a SpaCy-based ensemble, encompassing Bag of Words and Convolutional Neural Networks (CNN), is designed for text classification. Additionally, the integration of an LSTM model with Word Embedding via Keras and pad_Sequences has also been trained.
A careful assessment highlights that the SpaCy-based combination consistently achieves better F1 scores across various evaluation measures. Simultaneously, the LSTM-Word Embedding synergy manifests compelling results. Our finding suggests some combination of language models and classification techniques that can be used to automate the requirement review process and also try to evaluate the use of various word embedding techniques in the requirement review process.
Beskrivning
Ämne/nyckelord
Computer, science , software engineering , automotive , requirements , machine learning , language models , thesis