Master thesis- Fostering Appropriate Trust in Agentic Chatbots for Enterprise Use

Abstract

This thesis investigates challenges of developing agentic AI for fostering trust in enterprise users. Using Research through Design (RtD) as a methodological framework, an LLM-driven chatbot was iteratively designed and integrated into a web-based system for managing flexible office space, with the aim of automating data retrieval and analysis through natural language interaction. Two design iterations were developed and evaluated through semi-structured interviews, live chatbot trials, and trace based analysis of agent behavior. The studies involved enterprise users with direct operational responsibility on the platform, providing empirical grounding for identifying trust-related design challenges. Three interconnected challenges emerged from the process. First, a probabilistic/deterministic boundary: LLMs operating as probabilistic systems must produce strictly typed, correct database queries, requiring architectural decisions that progressively constrain where non-deterministic behavior can corrupt factual outputs. Second, a domain-model gap: users interact via natural language rooted in personal mental models and informal terminology, while the agent depends on exact entity names and structured filters — a structural mismatch that complicates relying on database queries. Third, early-stage trust was found to depend disproportionately on the consistent, verifiable correctness of simple operations; isolated failures on basic tasks damaged perceived reliability more than successes on complex ones could build it. These findings are discussed through established trust literature. Finally a teacher-pupil/manager-engineer framework is proposed to describe the stages of trust in AI deployment. The thesis concludes that verifiability-first design, transparent reasoning communication, and architectural containment of probabilistic behavior are prerequisite conditions for fostering long-term use of domain-specific agentic chatbots. Keywords: agentic AI, trust in AI, LLM applications, enterprise chat-bot, Research through Design, human-AI interaction