An AI Agent for Exploratory Testing Guidance Using Historical Fault Data - A Case Study on Ericsson MINI-LINK

Abstract

Exploratory testing is effective at finding defects that scripted tests miss, but it depends heavily on domain knowledge that is hard to transfer between testers. In telecommunications, where systems have thousands of configurable parameters, much of this knowledge stays locked in bug trackers and individual experience. This thesis explores how AI can support exploratory testing by making historical fault data searchable and actionable. The result is a conversational assistant, the ETAgent, developed and evaluated at Ericsson for the MINI-LINK microwave product family. The system builds a knowledge base from over 4,000 trouble reports. Unsupervised topic modelling discovers recurring fault patterns from verified reports, which are summarised into actionable guides. A hybrid retrieval pipeline combining sparse and dense search with cross-encoder reranking handles diverse query types, achieving an MRR of 0.815. A LangGraph-based agentic architecture lets the model choose retrieval strategies and synthesise guidance across multi-turn conversations. In a blind evaluation, eleven domain experts preferred the ETAgent over both a foundation model and an enterprise baseline in 72% of comparisons. The advantage was clearest on diagnosis and knowledge queries, where retrieval-grounded answers provide concrete ticket references that neither baseline can offer. The system’s value lies in surfacing specific historical evidence rather than general test planning advice, where the raw foundation model remains competitive.