Insight generation using language models in enterprise settings - From retrieval to recursion: a comparative enterprise case study

Abstract

Recent improvements in generative AI, in particular Large Language Models (LLM) have created a desire for organizations to utilize the technology to drive value cre ation. One promising application of the technology is insight generation from inter nal enterprise data. Despite rapid technological advancements, organizations face substantial technical, organizational, and environmental challenges when trying to implement such systems. The purpose of this thesis is to investigate insight gener ation in enterprise settings, and which key aspects and challenges that influence a successful implementation. The study is conducted as a single-case design research project within a large enter prise environment. Based on exploratory findings and requirement analysis, two lan guage model based pipelines are designed, implemented, and evaluated: a Retrieval Augmented Generation (RAG) pipeline with Chain-of-Thought prompting, and a Recursive Language Model (RLM) inspired pipeline, using iterative reasoning and content retrieval. Both pipelines are integrated into a chat-based prototype that al lows users to query a large internal corpus, retrieving results with clear provenance. The pipelines are evaluated using a combination of automated metrics and qual itative expert assessments. The results indicate that while both approaches are capable of generating insights from internal enterprise data, the RLM pipeline con sistently produces more relevant, and insightful responses, while also being more trustworthy. However, this increased performance comes at the cost of increased computational overhead in the form of latency and token usage. Beyond model accuracy, the findings suggest that successful enterprise adoption depends equally on non-technical factors, including trust, transparency, organizational constraints, and organizational readiness. These factors have to be addressed as requirements and implemented into the solutions artifact to ensure a successful insight generation system.