Multimodal Fusion of Pathology and Radiology Foundation Models for WHO 2021 Glioma Subtyping

MCML Authors

Daniel Scholz

→ Group Benedikt Wiestler
 AI for Image-Guided Diagnosis and Therapy

Benedikt Wiestler

Prof. Dr.

Principal Investigator

AI for Image-Guided Diagnosis and Therapy

Peter Schüffler

Prof. Dr.

Associate

Computational Pathology

Abstract

Molecular subtyping of gliomas is a common clinical task, yet challenging to perform on histology or radiology images alone. To address this challenge, we developed a multimodal classification framework that integrates histopathology and magnetic resonance imaging (MRI) using foundation models as unimodal experts, and evaluated three modality fusion strategies. Models are trained on two unpaired datasets of 772 histopathology cases and 959 multiparametric MRI scans, and tested on 171 unseen patient-matched cases. Multimodal models consistently outperform their unimodal counterparts, with a mixture-of-experts architecture achieving the strongest performance (AUC = 0.98 validation; AUC = 0.94 independent test set). Notably, we show that high-performing multimodal classifiers can be trained even without paired multimodal data, and in purely unimodal settings, these models match unimodal baselines. Finally, a detailed analysis of the learned multimodal representations reveals that the model identifies distinct visual biomarkers associated with glioma molecular subtypes, providing interpretable insight into its decision-making process.

article SSR+26

---

npjpo

10.118. Mar. 2026.

---

Authors

C. Saueressig • D. Scholz • P. Raffler • C. Delbridge • B. Wiestler • P. J. Schüffler

---

Links

DOI

---

Research Area

 C1 | Medicine