On the Interplay of Priors and Overparametrization in Bayesian Neural Network Posteriors

MCML Authors

Julius Kobialka

→ Group David Rügamer
 Statistics, Data Science and Machine Learning

Emanuel Sommer

→ Group David Rügamer
 Statistics, Data Science and Machine Learning

David Rügamer

Prof. Dr.

Principal Investigator

Statistics, Data Science and Machine Learning

Abstract

Bayesian neural network (BNN) posteriors are often considered impractical for inference, as symmetries fragment them, non-identifiabilities inflate dimensionality, and weight-space priors are seen as meaningless. In this work, we study how overparametrization and priors together reshape BNN posteriors and derive implications allowing us to better understand their interplay. We show that redundancy introduces three key phenomena that fundamentally reshape the posterior geometry: layer balancedness, weight distribution on equal-probability manifolds, and prior conformity. We validate our findings through extensive experiments with posterior sampling budgets that far exceed those of earlier works, and demonstrate how overparametrization induces structured, prior-aligned weight posterior distributions.

inproceedings KSK+25b

---

EIML @EurIPS 2025

Workshop on Epistemic Intelligence in Machine Learning at the European Conference on Information Processing Systems. Copenhagen, Denmark, Dec 03-05, 2025. To be published.

---

Authors

J. Kobialka • E. Sommer • J. Kwon • D. Dold • D. Rügamer

---

Research Area

 A1 | Statistical Foundations & Explainability