Home | Publications | SSM+26

Guiding Posterior Exploration With Optimizer-Derived Geometry

MCML Authors

Moritz Schlager

→ Group Daniel Rückert
Artificial Intelligence in Healthcare and Medicine

Emanuel Sommer

→ Group David Rügamer
Statistics, Data Science and Machine Learning

David Rügamer

Prof. Dr.

Core PI

Statistics, Data Science and Machine Learning

Abstract

Sampling-based methods offer a principled approach to uncertainty quantification in Bayesian neural networks. Their practical use, however, is often challenged by the computational cost of exploring high-dimensional and multimodal posterior distributions. To overcome these difficulties, Bayesian Deep Ensembles, i.e., warmstarting the sampling from several optimized solutions, have proven to be an effective strategy. In this paper, we demonstrate that curvature estimates computed during the warmstart as a byproduct in adaptive optimizers such as AdamW can inform the sampling phase at negligible additional cost. Specifically, our proposed preconditioned sampling strategy based on optimizer-derived geometries can substantially reduce or even eliminate the need for a lengthy sampling burn-in phase and leads to greater numerical stability. This approach consistently maintains or improves predictive performance and uncertainty quantification without any additional computational costs. We confirm the consistency of our findings across various datasets and network architectures.

inproceedings SSM+26

OPTIMAL @AISTATS 2026

Workshop on Optimisation and Post-Bayesian Inference in Machine Learning at the 29th International Conference on Artificial Intelligence and Statistics. Tangier, Morocco, May 02-05, 2026.