On the Epistemic Uncertainty of Overparametrized Neural Networks

MCML Authors

David Rügamer

Prof. Dr.

Core PI

Statistics, Data Science and Machine Learning

Abstract

Epistemic uncertainty is often viewed as a reducible uncertainty that vanishes with increasing data. This perspective implicitly assumes parameter identifiability and equates epistemic uncertainty with predictive variability. In overparametrized neural networks, however, model parameters are typically non-identifiable due to symmetries and redundant representations. As a consequence, substantial parameter uncertainty can persist even when the underlying function is fully identified. In this work, we analyze epistemic uncertainty through the lens of non-identifiability and characterize both discrete and continuous sources of residual uncertainty. Focusing on one-hidden-layer ReLU networks, we thoroughly analyze the resulting posterior structure and validate our theoretical insights through empirical studies.

inproceedings Rue26

---

ICML 2026

43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026. To be published. Preprint available.

---

Authors

D. Rügamer

---

Links

arXiv

---

Research Area

 A1 | Statistical Foundations & Explainability