Systematic Design and Integration of Large Language Model Tools for Engineering Analysis

Abstract

Generative AI, and more specifically large language models (LLMs), show great promise in further facilitating and streamlining analysis engineering work. In this thesis, the current limitations for the use of such technologies regarding the incorporation in engineering tasks are investigated. This investigation consists of a comprehensive literature study, in addition to nine interviews with engineers at GKN Aerospace in Trollhättan, Sweden. The results indicate a number of challenges. One being the importance of embedding internal company knowledge when leveraging the capacities of LLMs, while the use of non-local LLMs is linked to considerable issues when it comes to handling sensitive data. With the results of this investigative study as a foundation, an LLM-based tool meant to mitigate these identified issues is developed. The development is done using LangChain, such that the OpenAI API can be used in a Python environment. The developed software is focused on the manipulation, and extracting, of data in CNS files. A file format containing results from finite element simulations. Different agentic systems, leveraging different methods for knowledge embedding such as retrieval augmented generation (RAG), and post-training such as fine-tuning, are investigated. The various architectures are evaluated based on their efficiency and accuracy in solving tasks. The results indicate that LLM-based tools have great potential in the field. The top performing architecture based on this testing is incorporated into a sub-graph architecture, for which usability and validation are examined. The results for efficiency, accuracy, usability testing and validation imply the considerable potential of leveraging LLMs in the domain. Nevertheless, performance is not perfect and a number of considerations in such a development must be taken. The methods for knowledge embedding and post-training seemingly have great impact on the performance, and more sophisticated approaches within RAG and fine-tuning have potential to further improve performance.