Home | Publications | PJH+26

Addressing Benchmarking Gaps in Large Language Models for Health and Medicine With Dynamic Red-Teaming

MCML Authors

Link to Profile Christian Wachinger

Christian Wachinger

Prof. Dr.

Core PI

Artificial Intelligence in Medical Imaging

Link to Profile Benedikt Wiestler

Benedikt Wiestler

Prof. Dr.

Core PI

AI for Image-Guided Diagnosis and Therapy

Abstract

Large language models (LLMs) are increasingly used to answer health-related questions and support healthcare workflows, yet evidence for their safety still relies heavily on static benchmarks that can rapidly become obsolete or be optimized against. Here we introduce a Dynamic, Automatic and Systematic (DAS) red-teaming audit framework that continuously stress-tests LLMs for health across four safety-critical axes: robustness, privacy, bias and hallucination. Validated against board-certified clinicians, a suite of adversarial agents autonomously mutates health-related test cases to uncover vulnerabilities in real time. Applying DAS to 15 state-of-the-art LLMs revealed a profound gap between high static benchmark performance and low dynamic reliability—the ‘benchmarking gap’. Despite median MedQA accuracy exceeding 80%, 94% of previously correct answers failed under dynamic robustness testing. This brittleness generalized to the realistic, open-ended HealthBench dataset, where top-tier models exhibited failure rates exceeding 70%, suggesting that high scores on established static benchmarks may reflect superficial memorization. We observed similarly high failure rates across other domains: privacy leaks were elicited in 86% of scenarios, cognitive bias priming altered recommendations in 81% of fairness tests and hallucination rates exceeded 74% in widely used models. By converting LLM safety evaluation for health from a static checklist into a living adversarial audit, DAS provides a scalable framework for surfacing latent risks before such systems are deployed in consumer-facing health assistants and broader clinical workflows.

article PJH+26

Nature Health

Jul. 2026.

Authors

J. Pan • B. Jian • P. Hager • Y. Zhang • C. Liu • F. Jungmann • H. B. Li • J. Canisius • C. You • J. Wu • J. Zhu • F. Liu • Y. Liu • N. Bubeck • M. Knolle • C. Chen • C. Wachinger • Z. Gong • C. Ouyang • G. Kaissis • B. Wiestler • D. 

Links

DOI

Research Area

 C1 | Medicine

BibTeXKey: PJH+26

#p-rueckert #p-wachinger #p-wiestler
Back to Top