Second-Order Uncertainty Quantification: Variance-Based Measures

MCML Authors

Yusuf Sale

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Paul Hofman

→ Group Eyke Hüllermeier
 Artificial Intelligence and Machine Learning

Lisa Wimmer

Dr.

* Former Member

→ Group Bernd Bischl
 Statistical Learning and Data Science

Eyke Hüllermeier

Prof. Dr.

Core PI

Artificial Intelligence and Machine Learning

Thomas Nagler

Prof. Dr.

Core PI

Computational Statistics & Data Science

Abstract

Uncertainty quantification is a critical aspect of machine learning models, providing important insights into the reliability of predictions and aiding the decision-making process in real-world applications. This paper proposes a novel way to use variance-based measures to quantify uncertainty on the basis of second-order distributions in classification problems. A distinctive feature of the measures is the ability to reason about uncertainties on a class-based level, which is useful in situations where nuanced decision-making is required. Recalling some properties from the literature, we highlight that the variance-based measures satisfy important (axiomatic) properties. In addition to this axiomatic approach, we present empirical results showing the measures to be effective and competitive to commonly used entropy-based measures.

misc SHW+23

Preprint

Authors

Links

Research Areas

 A1 | Statistical Foundations & Explainability

 A3 | Computational Models