Explainable AI for Automatic Document Classification in Regulated Finance

Abstract

The increasing volume of digital documents in regulated financial environments has created significant challenges related to information security, regulatory compliance, and efficient information management. Financial institutions routinely process sensitive information, including internal business data, customer records, and regulatory documents, where incorrect handling or classification may result in legal, financial, and reputational consequences. Despite the importance of information classification, the process is often performed manually, making it inconsistent, time-consuming, and difficult to scale. These challenges motivate the need for automated and trustworthy document classification systems that can support regulated organizations while maintaining transparency and accountability. This thesis investigates the use of transformer-based language models for automatic document classification in regulated financial environments, with a particular focus on explainability and auditability. The study explores how contextual and semantic information within documents can be used to distinguish between different information sensitivity levels, including Public, Internal, Confidential, and Strictly Confidential classifications. To address privacy and regulatory constraints, the work utilizes a synthetic and semi-controlled dataset .generated using a controlled template-based synthetic document generation methodology with constrained vocabulary, document structures, and contextual patterns designed to reflect the structural and linguistic characteristics of financial documents while avoiding the use of sensitive real-world data. The proposed system is based on a fine-tuned transformer architecture combined with explainable artificial intelligence (XAI) techniques. Attention-based explanations and Integrated Gradients feature attribution methods are integrated into the classification pipeline to provide insight into the model’s decision making process. The explainability analysis investigates whether the generated explanations align with meaningful contextual indicators associated with document sensitivity and whether they can support transparency, trust, and compliance requirements within regulated financial settings. The experimental results demonstrate that transformer-based models can effectively learn contextual patterns related to information sensitivity within the controlled dataset while also providing interpretable explanations of classification decisions. The study further analyzes explanation consistency, confidence behaviour, robustness against external documents, and potential shortcut learning effects. Since both the training and evaluation data were generated using the same controlled template-based document generation methodology, the results should be interpreted within the context of this experimental setting. Although separate documents were used for training and evaluation, come from the same dataset share similar linguistic and structural characteristics. Therefore, further evaluation using independent datasets is required to assess the generalizability of the proposed approach. This work contributes to the growing field of explainable AI in regulated industries by demonstrating how modern natural language processing techniques can be combined with explainability methods to support secure, transparent, and trustworthy information classification in financial organizations, while also highlighting the importance of independent evaluation when using controlled and synthetic data.