Home  | Publications | OPB+26

Specialized Foundation Models for Intelligent Operating Rooms

MCML Authors

Abstract

Surgical procedures unfold in complex environments demanding coordination between surgical teams, tools, imaging and increasingly, intelligent robotic systems. While AI solutions like ChatGPT and Gemini have revolutionized language understanding and seen early adaptions in clinical diagnosis, they fall short in the safety-critical, multimodal setting of surgery. Ensuring safety and efficiency in ORs of the future requires intelligent systems, like surgical robots, smart instruments and digital copilots, capable of understanding complex activities and hazards. We introduce ORQA, a multimodal foundation model unifying visual, auditory, and structured data for holistic surgical understanding. ORQA’s question-answering framework empowers diverse tasks, serving as an intelligence core for surgical technologies. We benchmark ORQA against generalist vision-language models, and show that while they struggle to perceive surgical scenes, ORQA delivers substantially stronger, consistent performance. To meet diverse deployment needs, we design, and release a family of smaller ORQA models tailored to different computational requirements. This work establishes a foundation for the next wave of intelligent surgical solutions, enabling surgical teams and medical technology providers to create smarter and safer operating rooms.

article OPB+26

npj Digital Medicine

Apr. 2026.
Top Journal

Authors

E. Özsoy • C. Pellegrini • D. Bani-Harouni • K. Yuan • M. Keicher • N. Navab

Links

DOI

Research Area

 C1 | Medicine

BibTeXKey: OPB+26

#p-navab
Back to Top