Home  | Publications | RMG+25

GroundingDINO for Open-Set Lesion Detection in Medical Imaging

MCML Authors

Link to Profile Julia Schnabel PI Matchmaking

Julia Schnabel

Prof. Dr.

Principal Investigator

Computational Imaging and AI in Medicine

Abstract

Open-world anomaly detection is a task in which machine learning is well-positioned to advance cancer diagnosis, potentially leading to significantly improved survival rates. For a model to be used in clinical settings, it must demonstrate high performance, robustness, and generalisability. A common approach to achieving high generalisability is to incorporate information from broader representations within the model. In this work, we investigate the application of GroundingDINO to medical anomaly detection and localisation, evaluating both its overall performance and the influence of text prompts. We find that GroundingDINO outperforms the YOLOv11n model even with minimal use of contextual information. When exploring methods to introduce more contextual information, we observe that specifying the organ within the prompt improves closed-set performance on rarer lesion classes. However, adding visual descriptions of lesions during training leads to a significant performance drop on those subsets, indicating that the model memorises prompt-image pairs rather than learning meaningful semantic relationships. Our work highlights a critical limitation of GroundingDINO in medical imaging and proposes targeted modifications to the model architecture or training strategies as promising directions for utilising richer semantic prompts to improve anomaly detection.

inproceedings

MSB EMERGE @MICCAI 2025

2nd MICCAI Student Board Emerge Workshop at the 28th International Conference on Medical Image Computing and Computer Assisted Intervention. Daejeon, Republic of Korea, Sep 23-27, 2025. To be published. Preprint available.

Authors

S. J. Roughley • J. P. Müller • S. Gao • Z. Gao • M. Ligero • R. Blums • M. Crispin-Ortuzar • J. A. Schnabel • B. Kainz • C. I. Bercea • I. P. Machado

Links

URL

Research Area

 C1 | Medicine

BibTeXKey: RMG+25

Back to Top