Amortised Inference Through One-Step Implicit Sampling

MCML Authors

Vincent Pauline

→ Group Stefan Bauer
 Algorithmic Machine Learning & Explainable AI

Stefan Bauer

Prof. Dr.

Core PI

Algorithmic Machine Learning & Explainable AI

Abstract

We study the problem of amortised sampling from an energy-based distribution without access to samples from the target. Inspired by recent ideas in data-based generative modelling, we propose an algorithm for training a one-step implicit generator by turning invariance of the modelled distribution under a target-invariant MCMC kernel into a training signal that progressively corrects the generator. Each step of training forms an empirical approximation to the modelled distribution by sampling from the model, evolves it towards the target distribution via a short MCMC chain, and minimises a divergence between the two empirical distributions, thus encouraging the model to move in the direction of the MCMC kernel's evolution. In the asymptotic limit, the modelled distribution is a fixed point of the kernel and thus samples from the target. We study various design choices for the MCMC kernel, the divergence, and the training procedure and show that the proposed one-step implicit sampler (OSIS) performs surprisingly well compared to multi-step amortised samplers and non-amortised MCMC baselines while being far more efficient at sampling time.

inproceedings PTC+26

---

SPIGM @ICML 2026

Workshop on Structured Probabilistic Inference and Generative Modeling at the 43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026.

---

Authors

V. Pauline • K. Tamogashev • A. Carter • S. Choi • S. Bauer • E. S. Whitammer

---

Links

URL

---

Research Area

 A1 | Statistical Foundations & Explainability