Multimodal Classification of Adult-Type Diffuse Gliomas using Deep Learning on Whole-Slide Images and MRI

Abstract

Adult-type diffuse gliomas are the most common malignant brain tumors and accurate molecular classification is essential for diagnosis and treatment planning. This thesis investigates deep learning approaches for classifying IDH mutation status and 1p/19q codeletion status using H&E-stained whole-slide images (WSIs) and magnetic resonance imaging (MRI). In a first step, foundation models (FMs) were used to extract feature vectors from the images, which were subsequently used as input to the models that performed the final classification. Both unimodal and multimodal models were evaluated, where different multimodal fusion techniques were explored to combine histopathology and MRI features. The study was conducted on data from Sahlgrenska University Hospital, including 543 WSIs, 528 MRI scans and 525 multimodal patient pairs. Results showed that multimodal models achieved the best overall performance, with the highest test AUC of 0.965 for IDH classification and 0.987 for the codeletion classification task. WSI-based models consistently outperformed MRI-based models, while MRI provided complementary information that improved certain multimodal models. Furthermore, for the WSI-based models, attention heatmaps could be generated, which may improve interpretability and strengthen their potential clinical applicability. The findings demonstrate that deep learning and FMs can enable reliable molecular classification of adult-type diffuse gliomas, while multimodal models offer modest improvements over approaches based only on histopathology.