Natural Language Processing and Large Language Models for Automation of Compliance Tracing

Abstract

Compliance is a costly and time-consuming task that most, if not all, firms must perform. As such, automating parts of the compliance process could be highly valuable. This thesis aims to investigate challenges faced by European software-intensive firms in their compliance processes, identify automation opportunities, and develop a Natural Language Processing- and Large Language Model-based software artifact to automate compliance tracing between company guidelines and normative requirements. The thesis followed the Design Science Research approach, and as such, the research was conducted in close collaboration with industry practitioners. The challenges and automation opportunities were identified together with seven interviewees from four different companies, and the final software artifact, dubbed TraceAlign, was developed and evaluated in focus groups with a total of twelve unique participants from two companies. The identified challenges ranged from organizational- and management-related to specifics inherent to the specifications of normative requirements. Automation opportunities related mainly to the management of requirements, company guidelines, and compliance evidence, of which this thesis focuses specifically on the task of compliance tracing of company guidelines to normative requirements. The final software artifact, TraceAlign, was considered to be time- and cost-saving by the focus group participants, but could perhaps be made more accurate. We conclude that there are many challenges with compliance that could potentially be automated using Natural Language Processing and Large Language Models.