SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutions

MCML Authors

Jessica Bader

→ Group Zeynep Akata
 Interpretable and Reliable Machine Learning

Leander Girrbach

→ Group Zeynep Akata
 Interpretable and Reliable Machine Learning

Stephan Alaniz

Dr.

→ Group Zeynep Akata
 Interpretable and Reliable Machine Learning

Zeynep Akata

Prof. Dr.

Principal Investigator

Interpretable and Reliable Machine Learning

Abstract

Concept Bottleneck Models (CBMs) and other concept-based interpretable models show great promise for making AI applications more transparent, which is essential in fields like medicine. Despite their success, we demonstrate that CBMs struggle to reliably identify the correct concepts under distribution shifts. To assess the robustness of CBMs to concept variations, we introduce SUB: a fine-grained image and concept benchmark containing 38,400 synthetic images based on the CUB dataset. To create SUB, we select a CUB subset of 33 bird classes and 45 concepts to generate images which substitute a specific concept, such as wing color or belly pattern. We introduce a novel Tied Diffusion Guidance (TDG) method to precisely control generated images, where noise sharing for two parallel denoising processes ensures that both the correct bird class and the correct attribute are generated. This novel benchmark enables rigorous evaluation of CBMs and similar interpretable models, contributing to the development of more robust methods.

inproceedings

---

ICCV 2025

IEEE/CVF International Conference on Computer Vision. Honolulu, Hawai'i, Oct 19-23, 2025. To be published. Preprint available.

---

Authors

J. Bader • L. Girrbach • S. Alaniz • Z. Akata

---

Links

GitHub

---

Research Area

 B1 | Computer Vision