Design and Evaluation of a RAG Chatbot in an Industrial Setting

Abstract

The rapid advancements in large language models (LLMs) have sparked industrial interest in leveraging generative AI for information access and decision support. However, the limitations of standalone LLMs, such as hallucinations and a lack of domain-specific or up-to-date knowledge, pose significant barriers in enterprise settings. Retrieval-Augmented Generation (RAG) architectures offer a promising solution by integrating external document retrieval into the generative process. This thesis presents the design, implementation, and evaluation of a prototype RAG chatbot developed in collaboration with Volvo Group Trucks Technology (GTT). The chatbot enables users to interact conversationally with internal data comprising multiple modalities, including text and images. The system architecture was built from the ground up, focusing on document parsing, vector-based retrieval using FAISS, and multimodal integration. Evaluation covered functionality, generation quality, and usability, with positive feedback from end users supporting the system’s practical viability. This work contributes to applied research on RAG systems in industrial environ ments, highlighting challenges related to multimodal data handling, data security, and ethical considerations such as GDPR compliance. It serves both as a proof of concept for internal use and as a blueprint for broader AI adoption within the organization and similar industrial contexts.