Connecting the Dots: Is Mode Connectedness the Key to Feasible Sample-Based Inference in Bayesian Neural Networks?

MCML Authors

Emanuel Sommer

→ Group David Rügamer
 Statistics, Data Science and Machine Learning

Lisa Wimmer

Dr.

* Former Member

→ Group Bernd Bischl
 Statistical Learning and Data Science

Ludwig Bothmann

Dr.

→ Group Bernd Bischl
 Statistical Learning and Data Science

Bernd Bischl

Prof. Dr.

Director

Statistical Learning and Data Science

David Rügamer

Prof. Dr.

Core PI

Statistics, Data Science and Machine Learning

Abstract

A major challenge in sample-based inference (SBI) for Bayesian neural networks is the size and structure of the networks' parameter space. Our work shows that successful SBI is possible by embracing the characteristic relationship between weight and function space, uncovering a systematic link between overparameterization and the difficulty of the sampling problem. Through extensive experiments, we establish practical guidelines for sampling and convergence diagnosis. As a result, we present a Bayesian deep ensemble approach as an effective solution with competitive performance and uncertainty quantification.

inproceedings SWP+24

---

ICML 2024

41st International Conference on Machine Learning. Vienna, Austria, Jul 21-27, 2024.

---

Authors

E. Sommer • L. Wimmer • T. Papamarkou • L. Bothmann • B. Bischl • D. Rügamer

---

Links

URL

---

Research Area

 A1 | Statistical Foundations & Explainability