Vectorising Tabular Data for RAG and Vector Database Validation - A study of tabular data retrieval and anomaly detection methods against poisoning attacks in RAG

Abstract

Current Retrieval-Augmented Generation (RAG) systems are primarily optimised for unstructured text and lack mechanisms for ensuring data integrity in the knowledge base. This thesis investigates two challenges in the context of SVEA, AI Sweden’s digital assistant for the Swedish public sector. Firstly, how structured tabular data should be represented for vector retrieval. Secondly, how a validation layer can protect the vector database against data poisoning attacks. The two attacks tested are prompt injections and hate speech.

Four table representation methods are evaluated on a corpus of 23,638 chunks from Swedish government and municipality documents. The methods are the following: Unmodified Tabular Extraction (UTE), Metadata-Enriched Representation (MER), Natural Language Summarization (NLS), and Summary-Augmented Hybrid Representation (SAHR). The results indicate that SAHR achieves the highest table Recall@1, Recall@3 and MRR (0.362, 0.496 and 0.445), while UTE achieves the highest Recall@10 (0.630). This shows a trade-off between ranking quality and retrieval coverage. In addition, the cost in terms of latency of generating summaries with an LLM must be considered.

Unsupervised anomaly detection using HDBSCAN and Isolation Forest is combined with pre-trained classifiers for prompt injection and hate speech detection as a potential validation layer. Using only classifiers achieves the highest F1-score (0.592 for prompt injection on Regeringen), while hybrid two-stage approaches reduce latency at the cost of reduced detection performance.