Home | Publications | SMS+26

The Impact of Multisensory Feedback on Supporting Physicians in Robotic Ultrasound Interaction

MCML Authors

Abstract

Recent robotics advancements have enabled novel applications in medicine, such as automating ultrasound acquisition to support sonographers and enable remote operation. However, a key challenge is the lack of haptic and perceptual transparency regarding the robot’s decisions and actions, which may undermine trust and user acceptance. In this study, we propose a multisensory feedback system to enhance physician–robot interaction during robotic ultrasound procedures. The system provides real-time feedback on the force exerted by the robot on the patient and was developed following a user-centered approach, informed by clinical domain expertise, to support usability and clinical relevance. In a simulated Extended Reality (XR) environment, we evaluated the impact of different feedback conditions—including two sonification strategies, one visual feedback method, and their combinations. A user study with 30 participants, including three practicing clinicians, was conducted to assess these modalities. Results showed that multisensory feedback significantly reduced cognitive load (NASA-TLX: A1+V vs.V, p<0.001; A2+V vs. V, p < 0.01), improved usability (SUS: A1+V vs. V, p < 0.001; A2+V vs. V, p < 0.001) and perceived ease of use (SEQ: A1+V vs. V, p < 0.01; A2+V vs. V, p < 0.01). While visual feedback alone yielded better performance in force monitoring, the multisensory conditions supported better focus on the primary task of image analysis. These findings suggest that multisensory feedback can enhance user experience and support interaction factors associated with trust formation, such as reduced workload and improved awareness of robot behavior, especially with further training and adaptation. Given the predominantly non-clinical sample and the use of abstract visual targets, these results should be read as an exploratory, preclinical validation of multisensory human–robot interaction rather than as evidence of clinical efficacy.

article SMS+26

IEEE Access

Early Access. Jul. 2026.
Top Journal

Authors

T. Song • S. Matinfar • L. Schütz • F. Li • U. Eck • N. Navab

Links

DOI

Research Area

 C1 | Medicine

BibTeXKey: SMS+26

#p-navab
Back to Top