CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control and Altering of T2I Models

MCML Authors

Nick Stracke

→ Group Björn Ommer
 Computer Vision & Learning

Björn Ommer

Prof. Dr.

Core PI

Computer Vision & Learning

Abstract

Text-to-image generative models have become a prominent and powerful tool that excels at generating high-resolution realistic images. However, guiding the generative process of these models to take into account detailed forms of conditioning reflecting style and/or structure information remains an open problem. In this paper, we present. LoRAdapter, an approach that unifies both style and structure conditioning under the same formulation using a novel conditional LoRA block that enables zero-shot control. LoRAdapter is an efficient and powerful approach to condition text-to-image diffusion models, which enables fine-grained control conditioning during generation and outperforms recent state-of-the-art approaches.

inproceedings SBS+24a

---

ECCV 2024

18th European Conference on Computer Vision. Milano, Italy, Sep 29-Oct 04, 2024.

---

Authors

N. Stracke • S. A. Baumann • J. M. Susskind • M. A. Bautista • B. Ommer

---

Links

DOI GitHub

---

In Collaboration

 Apple

---

Research Area

 B1 | Computer Vision