Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences

MCML Authors

Viktor Stein

Dr.

→ Group Massimo Fornasier
 Applied Numerical Analysis

Abstract

Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes. Using a unified information-geometric framework, we introduce KL analogs by replacing the Fisher–Rao geometry in the dynamical formulation of the KL with transport-based geometries, and derive closed-form expressions for common distribution families. Between elliptic distributions, these divergences remain finite for degenerating equal covariances and yield a geometric interpretation of regularization heuristics used in Kalman ensemble methods. We demonstrate the utility of these divergences in KL-regularized optimal control. In the fully tractable setting of linear time-invariant systems with Gaussian process noise, the classical KL reduces to a quadratic control penalty that becomes singular as process noise vanishes. Our variants remove this singularity and yield well-posed problems. On a double integrator and a cart-pole example, the resulting controls preserve nontrivial feedback and achieve better closed-loop performance.

inproceedings SDA26

---

ICML 2026

43rd International Conference on Machine Learning. Seoul, South Korea, Jul 06-11, 2026. To be published. Preprint available.

---

Authors

V. Stein • J. M. de Frutos

---

Links

URL GitHub

---

Research Area

 A2 | Mathematical Foundations