Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation

MCML Authors

Felix Krause

→ Group Björn Ommer
 Computer Vision & Learning

Björn Ommer

Prof. Dr.

Core PI

Computer Vision & Learning

Abstract

Text-to-image diffusion models have two major drawbacks that severely limit their practical utility: (1) standard models lack an intrinsic mechanism for continuous, concept-specific guidance (e.g., for precisely controlling how aesthetically pleasing an image looks), and (2) they lack reliability for tasks requiring high local coherence (e.g., generating text or human hands). To tackle these issues, we introduce a novel notion of concept-wise mutual information and find large, concept-dependent differences between individual layers, demonstrating that the generation of specific structures is localized in distinct parts of the network. We exploit this insight by reinforcing the impact of concept-relevant layers in Concept Guidance (CoG), a precise, target-specific guidance method that works for models out-of-the-box without additional training, external models, gradients, or prompt engineering. CoG first quantifies each layer's concept-specific impact and then guides denoising using a weighted combination of predictions generated with concept-relevant layers skipped. We demonstrate performance increases across various targets and popular models like PixArt-alpha, SD3, SD3.5, and FLUX.1-dev.

inproceedings RHK+26

---

GCPR 2026

German Conference on Pattern Recognition. Siegen, Germany, Sep 22-25, 2026. To be published. Preprint available.

---

Authors

N. Röhrich • I. Hans • F. Krause • B. Ommer

---

Links

arXiv GitHub

---

Research Area

 B1 | Computer Vision