Towards Quantifying Incompatibilities in Evaluation Metrics for Feature Attributions

MCML Authors

Thomas Decker

→ Group Volker Tresp
 Database Systems, Data Mining and AI

Volker Tresp

Prof. Dr.

Principal Investigator

Database Systems, Data Mining and AI

Abstract

Feature attribution methods are widely used to explain machine learning models, yet their evaluation is challenging due to competing quality criteria such as faithfulness, robustness, and sparsity. These criteria often conflict, and even alternative formulations of the same metric can yield inconsistent conclusions. We address this by introducing a unifying framework that analyzes systematic incompatibilities between measures of explanation quality. Within this framework, we develop two novel mathematical tools: a samplewise incompatibility index that quantifies systematic conflicts between criteria, and a generalized eigen-analysis that localizes where tradeoffs are concentrated within attribution results. Experiments on image classifiers show that this analysis provides insights beyond isolated metrics and complements current evaluation practices for feature attributions.

inproceedings DT26

---

XAI4Science @AAAI 2026

2nd Workshop XAI4Science: From Understanding Model Behavior to Discovering New Scientific Knowledge at the 40th Conference on Artificial Intelligence. Singapore, Jan 20-27, 2026. To be published.

---

Authors

T. Decker • V. Tresp

---

Research Area

 A3 | Computational Models