Supporting the generation of engineering analysis reports with Large Language Models
Publicerad
Författare
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
At GKN Aerospace Sweden, engineers work hard to conduct simulations and analyses
to verify the performance of their products. Being an aerospace company, the
main product portfolio consists of aero-engine components, space systems and upgrades
of military aircraft engines. The outcome of the analysis performed on these
types of products needs to be documented in technical analysis reports. To go from
the generation of a report to the final stamp of approval, 3 roles are involved: Author,
reviewer and approver. The process often has long lead times and suffers from
many unnecessary loops and iterations back and fourth between the roles. With
the rapid rise of AI in the form of Large Language Models, there is an interest in
evaluating how this technology can be used to aid the engineers in this process and
reduce lead time. This is the main aim of this study. An important factor is that
due to confidentiality, data from the company cannot be transferred to unauthorized
LLM providers. This study conducts a thorough interview study with engineers responsible
for the different parts of the process. A qualitative thematic analysis of
the interview transcripts then resulted in a comprehensive process problem analysis
clearly mapping the issues within. Literature research was carried out providing a
wide overview of the recent advancements in the field of Large Language Models and
inspiring implementations. Using the problem analysis and literature, several design
criteria were defined and explored based on how the process can be supported using
LLMs. Software was developed using the Langchain framework in Python, resulting
in 12 different concepts to support the process. The outcome shows that LLM
has the potential to generate reviews of the engineering analysis reports, producing
promising results in five test cases created regarding the topics language, content,
references and consistency. However, the study finds that in the cases where the
concepts found a high proportion of report errors, there were also problems with
the production of a lot of false or untruthful feedback. Further demonstrations of
generating or retrieving parts of a report were developed and are presented as proofof-
concept. Lastly, a prototype of a user interface visualises how GKN Aerospace
engineers might interact with the developed functionality. The study strongly indicates
that GKN Aerospace can benefit to a great extent from such solutions. All code
implementations can be found at https://github.com/DaniSode/Master_Thesis.
Beskrivning
Ämne/nyckelord
Large Language Models, LLM, Engineering, Support, Reasoning, Reflection, Multi-agent