Discrete Diffusion Models With Interpolation-Based Controllable Resampling

MCML Authors

Sirine Ayadi

→ Group Stephan Günnemann
 Data Analytics & Machine Learning

Marcel Kollovieh

→ Group Stephan Günnemann
 Data Analytics & Machine Learning

Stephan Günnemann

Prof. Dr.

Core PI

Data Analytics & Machine Learning

Abstract

Discrete diffusion models form a powerful class of generative models across diverse data modalities, including graphs and text. However, existing approaches face fundamental limitations. Masked diffusion models suffer from irreversible errors if tokens are unmasked too early in the reverse process, while uniform diffusion models, despite enabling self-correction, often yield low-quality samples due to their strong reliance on intermediate latent states. To address these limitations, we introduce IDDM, an Interpolation-Based Discrete Diffusion Model that improves diffusion by reducing dependence on intermediate latent states. Central to IDDM is a controllable resampling mechanism that partially resets probability mass to the prior distribution, mitigating error accumulation and enabling token corrections. IDDM defines a generative process whose transitions interpolate between staying at the current state, resampling from a prior, and flipping toward the target state, while enforcing marginal consistency and fully decoupling training from inference. We benchmark our model against state-of-the-art discrete diffusion models on molecular graph and text generation tasks, showing competitive performance.

inproceedings AKG26

---

BeNTo @NeurIPS 2026

Workshop Beyond Next Token Prediction: Diffusion and Flow Models for Next-Generation Decoding at the 40th Conference on Neural Information Processing Systems. Sydney, Australia, Dec 06-12, 2026. To be published. Preprint available.

---

Authors

S. Ayadi • M. Kollovieh • S. Günnemann

---

Links

URL

---

Research Area

 A3 | Computational Models