PromptVFX: Text-Driven Fields for Open-World 3D Gaussian Animation

MCML Authors

Mert Kiray

→ Group Benjamin Busam
 Photogrammetry and Remote Sensing

Nassir Navab

Prof. Dr.

Core PI

Computer Aided Medical Procedures & Augmented Reality

Benjamin Busam

Prof. Dr.

Core PI

Photogrammetry and Remote Sensing

Abstract

Visual effects (VFX) are key to immersion in modern films, games, and AR/VR. Creating 3D effects requires specialized expertise and training in 3D animation software and can be time consuming. Generative solutions typically rely on computationally intense methods such as diffusion models which can be slow at 4D inference. We reformulate 3D animation as a field prediction task and introduce a text-driven framework that infers a time-varying 4D flow field acting on 3D Gaussians. By leveraging large language models (LLMs) and vision-language models (VLMs) for function generation, our approach interprets arbitrary prompts (e.g., “make the vase glow orange, then explode”) and instantly updates color, opacity, and positions of 3D Gaussians in real time. This design avoids overheads such as mesh extraction, manual or physics-based simulations and allows both novice and expert users to animate volumetric scenes with minimal effort on a consumer device even in a web browser11Web Demo: http://promptvfx.duckdns.org. Experimental results show that simple textual instructions suffice to generate compelling time-varying VFX, reducing the manual effort typically required for rigging or advanced modeling. We thus present a fast and accessible pathway to language-driven 3D content creation that can pave the way to democratize VFX further.

inproceedings KUN+26

---

3DV 2026

13th International Conference on 3D Vision. Vancouver, Canada, Mar 20-23, 2026.

---

Authors

M. Kiray • P. Uhlenbruck • N. Navab • B. Busam

---

Links

DOI GitHub

---

Research Areas

 B1 | Computer Vision

 C1 | Medicine