From Pixels to 3D Motion: Recreating the Physical Natural World From Images

On Monday, March 17th, 2025 at 2pm, Prof. Dr. Elliott (Shangzhe) Wu is visiting us from the University of Cambridge and giving a talk.

You can sign up for 1-on-1 meetings with him here.

Pixel-based generative models nowadays excel at creating compelling images, but often struggle to preserve basic physical properties, such as shape, motion, material, lighting, etc. These are often critical pieces bridging computer vision towards a wide range of real-world engineering applications, from interactive VR, robotics, design, and manufacturing, to scientific domains like biology and medical analysis. One fundamental challenge in computer vision is shifting from modeling pixel distributions to modeling physics-grounded representations, and characterizing 3D motion is a key stepping stone along this path. In this talk, I will mainly discuss a line of research that attempts to model dynamic 3D objects from casually recorded, in-the-wild images and videos, without any direct 3D supervision--an approach applicable to various natural objects like wildlife. The resulting model can then turn a single image into an animatable 3D asset in feed-forward fashion and generate 3D animations instantly.

Elliott (Shangzhe) Wu is an Assistant Professor in the Department of Engineering at the University of Cambridge. He received his PhD from the University of Oxford, advised by Andrea Vedaldi and Christian Rupprecht, and worked as a postdoc at Stanford University with Jiajun Wu. His research focuses on 3D computer vision and inverse graphics. His work received several awards, including the Best Paper Award at CVPR 2020, the BMVA Sullivan Doctoral Thesis Prize, and the ELLIS PhD Award.

Organized by:

Daniel Cremers
MCML

Almut Sophia Koepke
MCML