Label-Wise Aleatoric and Epistemic Uncertainty Quantification

MCML Authors

Yusuf Sale

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Paul Hofman

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Timo Löhr

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Lisa Wimmer

Dr.

* Former Member

→ Group Bernd Bischl
 Statistical Learning and Data Science

Thomas Nagler

Prof. Dr.

Core PI

Computational Statistics & Data Science

Eyke Hüllermeier

Prof. Dr.

Core PI

Artificial Intelligence and Machine Learning

Abstract

We present a novel approach to uncertainty quantification in classification tasks based on label-wise decomposition of uncertainty measures. This label-wise perspective allows uncertainty to be quantified at the individual class level, thereby improving cost-sensitive decision-making and helping understand the sources of uncertainty. Furthermore, it allows to define total, aleatoric, and epistemic uncertainty on the basis of non-categorical measures such as variance, going beyond common entropy-based measures. In particular, variance-based measures address some of the limitations associated with established methods that have recently been discussed in the literature. We show that our proposed measures adhere to a number of desirable properties. Through empirical evaluation on a variety of benchmark data sets – including applications in the medical domain where accurate uncertainty quantification is crucial – we establish the effectiveness of label-wise uncertainty quantification.

inproceedings SHL+24

---

UAI 2024

40th Conference on Uncertainty in Artificial Intelligence. Barcelona, Spain, Jul 16-18, 2024.

---

Authors

Y. Sale • P. Hofman • T. Löhr • L. Wimmer • T. Nagler • E. Hüllermeier

---

Links

URL

---

Research Areas

 A1 | Statistical Foundations & Explainability

 A3 | Computational Models