Explainable AI in Healthcare: Physicians Perspectives and Technical Evaluation of AI-Based Decision Support and Explainability Methods

Abstract

Artificial intelligence (AI) is increasingly integrated into healthcare, particularly in clinical decision support and predictive modeling. However, the limited interpretability of many machine learning models remains a major challenge for clinical implementation, motivating growing interest in explainable artificial intelligence (XAI). This thesis investigates XAI in healthcare from both technical and clinical perspectives. The clinical perspective was explored through qualitative interviews with physicians focusing on AI-based decision support systems and the role of explainable AI in clinical practice. The findings revealed a cautiously optimistic view of AI-supported decision-making, while emphasizing that clinically useful explanations should be concise, intuitive, and seamlessly integrated into existing workflows. The technical part of the study investigated XAI methods for survival prediction in lymphoma patients using both tabular clinical data and medical imaging data. Multiple survival modelling approaches, including Cox regression, DeepSurv, and convolutional neural network models, were implemented and evaluated using several post-hoc explainability methods across the different data modalities. While both modalities demonstrated strong predictive performance, the tabular models achieved slightly stronger results with more stable, interpretable explanations. Furthermore, different XAI approaches highlighted complementary but inconsistent patterns, illustrating challenges related to the robustness and reliability of post-hoc explanations. Overall, the findings demonstrated that successful clinical integration of AI depends as much on providing reliable, clinically meaningful explanations as it does on achieving strong predictive performance.