Research Group Daniel Cremers
Daniel Cremers
holds the Chair for Computer Vision and Artificial Intelligence at TU Munich since 2009.
In 2002 he obtained a PhD in Computer Science from the University of Mannheim, Germany. Subsequently he spent two years as a postdoctoral researcher at the University of California, Los Angeles (UCLA) and one year as a permanent researcher at Siemens Corporate Research in Princeton, NJ. From 2005 until 2009 he was associate professor at the University of Bonn, Germany. In 2016, Prof. Cremers received the Gottfried Wilhelm Leibniz Award, the biggest award in German academia.
Team members @MCML
Publications @MCML
DiffCD: A Symmetric Differentiable Chamfer Distance for Neural Implicit Surface Fitting.
18th European Conference on Computer Vision (ECCV 2024). Milano, Italy, Sep 29-Oct 04, 2024. To be published. Preprint at arXiv. arXiv. GitHub.
Abstract
Neural implicit surfaces can be used to recover accurate 3D geometry from imperfect point clouds. In this work, we show that state-of-the-art techniques work by minimizing an approximation of a one-sided Chamfer distance. This shape metric is not symmetric, as it only ensures that the point cloud is near the surface but not vice versa. As a consequence, existing methods can produce inaccurate reconstructions with spurious surfaces. Although one approach against spurious surfaces has been widely used in the literature, we theoretically and experimentally show that it is equivalent to regularizing the surface area, resulting in over-smoothing. As a more appealing alternative, we propose DiffCD, a novel loss function corresponding to the symmetric Chamfer distance. In contrast to previous work, DiffCD also assures that the surface is near the point cloud, which eliminates spurious surfaces without the need for additional regularization. We experimentally show that DiffCD reliably recovers a high degree of shape detail, substantially outperforming existing work across varying surface complexity and noise levels.
MCML Authors
GlobalPointer: Large-Scale Plane Adjustment with Bi-Convex Relaxation.
18th European Conference on Computer Vision (ECCV 2024). Milano, Italy, Sep 29-Oct 04, 2024. To be published. Preprint at arXiv. arXiv.
MCML Authors
MeshFeat: Multi-Resolution Features for Neural Fields on Meshes.
18th European Conference on Computer Vision (ECCV 2024). Milano, Italy, Sep 29-Oct 04, 2024. To be published. Preprint at arXiv. arXiv.
MCML Authors
Power Variable Projection for Initialization-Free Large-Scale Bundle Adjustment.
18th European Conference on Computer Vision (ECCV 2024). Milano, Italy, Sep 29-Oct 04, 2024. To be published. Preprint at arXiv. arXiv.
MCML Authors
MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation.
18th European Conference on Computer Vision (ECCV 2024). Milano, Italy, Sep 29-Oct 04, 2024. To be published. Preprint at arXiv. arXiv.
MCML Authors
Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model.
62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024). Bangkok, Thailand, Aug 11-16, 2024. URL.
MCML Authors
Variational Learning is Effective for Large Deep Networks.
41st International Conference on Machine Learning (ICML 2024). Vienna, Austria, Jul 21-27, 2024. URL. GitHub.
MCML Authors
Sparse Views, Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
Abstract
Neural approaches have shown a significant progress on camera-based reconstruction. But they require either a fairly dense sampling of the viewing sphere, or pre-training on an existing dataset, thereby limiting their generalizability. In contrast, photometric stereo (PS) approaches have shown great potential for achieving high-quality reconstruction under sparse viewpoints. Yet, they are impractical because they typically require tedious laboratory conditions, are restricted to dark rooms, and often multi-staged, making them subject to accumulated errors. To address these shortcomings, we propose an end-to-end uncalibrated multi-view PS frameworkfor reconstructing high-resolution shapes acquiredfrom sparse viewpoints in a real-world environment. We relax the dark room assumption, and allow a combination of static ambient lighting and dynamic near LED lighting, thereby enabling easy data capture outside the lab. Experimental validation confirms that it outperforms existing baseline approaches in the regime of sparse viewpoints by a large margin. This allows to bring high-accuracy 3D reconstruction from the dark room to the real world, while maintaining a reasonable data capture complexity.
MCML Authors
Partial-to-Partial Shape Matching with Geometric Consistency.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
MCML Authors
Boosting Self-Supervision for Single-View Scene Completion via Knowledge Distillation.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
MCML Authors
Finsler-Laplace-Beltrami Operators with Application to Shape Analysis.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
MCML Authors
Flattening the Parent Bias: Hierarchical Semantic Segmentation in the Poincaré Ball.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
MCML Authors
Cache Me if You Can: Accelerating Diffusion Models through Block Caching.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. DOI.
MCML Authors
A Perspective on Deep Vision Performance with Standard Image and Video Codecs.
Workshop at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). Seattle, WA, USA, Jun 17-21, 2024. PDF.
MCML Authors
HoloNets: Spectral Convolutions do extend to Directed Graphs.
12th International Conference on Learning Representations (ICLR 2024). Vienna, Austria, May 07-11, 2024. URL.
Abstract
Within the graph learning community, conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs: Only there could the existence of a well-defined graph Fourier transform be guaranteed, so that information may be translated between spatial- and spectral domains. Here we show this traditional reliance on the graph Fourier transform to be superfluous and -- making use of certain advanced tools from complex analysis and spectral theory -- extend spectral convolutions to directed graphs. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based. In order to thoroughly test the developed theory, we conduct experiments in real world settings, showcasing that directed spectral convolutional networks provide new state of the art results for heterophilic node classification on many datasets and -- as opposed to baselines -- may be rendered stable to resolution-scale varying topological perturbations.
MCML Authors
An Analytical Solution to Gauss-Newton Loss for Direct Image Alignment.
12th International Conference on Learning Representations (ICLR 2024). Vienna, Austria, May 07-11, 2024. URL.
Abstract
Direct image alignment is a widely used technique for relative 6DoF pose estimation between two images, but its accuracy strongly depends on pose initialization. Therefore, recent end-to-end frameworks increase the convergence basin of the learned feature descriptors with special training objectives, such as the Gauss-Newton loss. However, the training data may exhibit bias toward a specific type of motion and pose initialization, thus limiting the generalization of these methods. In this work, we derive a closed-form solution to the expected optimum of the Gauss-Newton loss. The solution is agnostic to the underlying feature representation and allows us to dynamically adjust the basin of convergence according to our assumptions about the uncertainty in the current estimates. These properties allow for effective control over the convergence in the alignment process. Despite using self-supervised feature embeddings, our solution achieves compelling accuracy w.r.t. the state-of-the-art direct image alignment methods trained end-to-end with pose supervision, and demonstrates improved robustness to pose initialization. Our analytical solution exposes some inherent limitations of end-to-end learning with the Gauss-Newton loss, and establishes an intriguing connection between direct image alignment and feature-matching approaches.
MCML Authors
S4C: Self-Supervised Semantic Scene Completion with Neural Fields.
11th International Conference on 3D Vision (3DV 2024). Davos, Switzerland, Mar 18-21, 2024. DOI.
MCML Authors
SupeRVol: Super-Resolution Shape and Reflectance Estimation in Inverse Volume Rendering.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024). Waikoloa, Hawaii, Jan 04-08, 2024. DOI.
Abstract
We propose an end-to-end inverse rendering pipeline called SupeRVol that allows us to recover 3D shape and material parameters from a set of color images in a superresolution manner. To this end, we represent both the bidirectional reflectance distribution function’s (BRDF) parameters and the signed distance function (SDF) by multi-layer perceptrons (MLPs). In order to obtain both the surface shape and its reflectance properties, we revert to a differentiable volume renderer with a physically based illumination model that allows us to decouple reflectance and lighting. This physical model takes into account the effect of the camera’s point spread function thereby enabling a reconstruction of shape and material in a super-resolution quality. Experimental validation confirms that SupeRVol achieves state of the art performance in terms of inverse rendering quality. It generates reconstructions that are sharper than the individual input images, making this method ideally suited for 3D modeling from low-resolution imagery.
MCML Authors
Masked Event Modeling: Self-Supervised Pretraining for Event Cameras.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024). Waikoloa, Hawaii, Jan 04-08, 2024. DOI.
MCML Authors
FIRe: Fast Inverse Rendering Using Directional and Signed Distance Functions.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024). Waikoloa, Hawaii, Jan 04-08, 2024. DOI.
MCML Authors
Deep Event Visual Odometry.
Preprint at arXiv (Dec. 2023). arXiv.
MCML Authors
GPT4MR: Exploring GPT-4 as an MR Sequence and Reconstruction Programming Assistant.
39th Annual Meeting of the European Society for Magnetic Resonance in Medicine and Biology (ESMRMB 2023). Basel, Switzerland, Oct 04-07, 2023. URL.
MCML Authors
To adapt or not to adapt? Real-time adaptation for semantic segmentation.
IEEE/CVF International Conference on Computer Vision (ICCV 2023). Paris, France, Oct 02-06, 2023. DOI.
MCML Authors
ΣIGMA: Quantum Scale-Invariant Global Sparse Shape Matching.
IEEE/CVF International Conference on Computer Vision (ICCV 2023). Paris, France, Oct 02-06, 2023. DOI.
MCML Authors
DDIT: Semantic Scene Completion via Deformable Deep Implicit Templates.
IEEE/CVF International Conference on Computer Vision (ICCV 2023). Paris, France, Oct 02-06, 2023. DOI.
MCML Authors
CASSPR: Cross Attention Single Scan Place Recognition.
IEEE/CVF International Conference on Computer Vision (ICCV 2023). Paris, France, Oct 02-06, 2023. DOI.
MCML Authors
Robust Autonomous Vehicle Pursuit without Expert Steering Labels.
IEEE Robotics and Automation Letters 8.10 (Oct. 2023). DOI.
MCML Authors
Multi Agent Navigation in Unconstrained Environments Using a Centralized Attention Based Graphical Neural Network Controller.
26th IEEE International Conference on Intelligent Transportation (ITSC 2023). Bilbao, Spain, Sep 24-28, 2023. DOI.
MCML Authors
LiDAR View Synthesis for Robust Vehicle Navigation Without Expert Labels.
26th IEEE International Conference on Intelligent Transportation (ITSC 2023). Bilbao, Spain, Sep 24-28, 2023. DOI.
MCML Authors
Geometrically Consistent Partial Shape Matching.
Preprint at arXiv (Sep. 2023). arXiv.
MCML Authors
SCP: Scene Completion Pre-training for 3D Object Detection.
Preprint at arXiv (Sep. 2023). arXiv.
MCML Authors
Beyond In-Domain Scenarios: Robust Density-Aware Calibration.
40th International Conference on Machine Learning (ICML 2023). Honolulu, Hawaii, Jul 23-29, 2023. URL.
MCML Authors
G-MSM: Unsupervised Multi-Shape Matching with Graph-based Affinity Priors.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada, Jun 18-23, 2023. DOI.
MCML Authors
Semidefinite Relaxations for Robust Multiview Triangulation.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada, Jun 18-23, 2023. DOI.
MCML Authors
Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada, Jun 18-23, 2023. DOI.
MCML Authors
Power Bundle Adjustment for Large-Scale 3D Reconstruction.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada, Jun 18-23, 2023. DOI.
MCML Authors
Behind the Scenes: Density Fields for Single View Reconstruction.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada, Jun 18-23, 2023. DOI.
MCML Authors
Non-Separable Multi-Dimensional Network Flows for Visual Computing.
Poster at the 44th Annual Conference of the European Association for Computer Graphics (EG 2023). Saarbrücken, Germany, May 08-12, 2023. DOI.
MCML Authors
Joint MR sequence optimization beats pure neural network approaches for spin-echo MRI super-resolution.
Preprint at arXiv (May. 2023). arXiv.
MCML Authors
Scale-Equivariant Deep Learning for 3D Data.
Preprint at arXiv (Apr. 2023). arXiv.
MCML Authors
E-NeRF: Neural Radiance Fields from a Moving Event Camera.
IEEE Robotics and Automation Letters 8.3 (Mar. 2023). DOI.
MCML Authors
High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023). Waikoloa, Hawaii, Jan 03-07, 2023. DOI.
MCML Authors
What Makes Graph Neural Networks Miscalibrated?.
36th Conference on Neural Information Processing Systems (NeurIPS 2022). New Orleans, LA, USA, Nov 28-Dec 09, 2022. PDF.
MCML Authors
Deep Combinatorial Aggregation.
36th Conference on Neural Information Processing Systems (NeurIPS 2022). New Orleans, LA, USA, Nov 28-Dec 09, 2022. PDF.
MCML Authors
A Graph Is More Than Its Nodes: Towards Structured Uncertainty-Aware Learning on Graphs.
Workshop on New Frontiers in Graph Learning at the 36th Conference on Neural Information Processing Systems (NeurIPS 2022). New Orleans, LA, USA, Nov 28-Dec 09, 2022. URL.
MCML Authors
Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration.
17th European Conference on Computer Vision (ECCV 2022). Tel Aviv, Israel, Oct 23-27, 2022. DOI.
MCML Authors
Biologically Inspired Neural Path Finding.
15th International Conference on Brain Informatics (BI 2022). Padova, Italy, Jul 15-15, 2022. DOI.
MCML Authors
The Probabilistic Normal Epipolar Constraint for Frame-To-Frame Rotation Optimization under Uncertain Feature Positions.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022). New Orleans, LA, USA, Jun 19-24, 2022. DOI.
MCML Authors
Lateral Ego-Vehicle Control Without Supervision Using Point Clouds.
3rd International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI 2022). Paris, France, Jun 01-03, 2022. DOI.
MCML Authors
Challenger: Training with Attribution Maps.
Preprint at arXiv (May. 2022). arXiv.
MCML Authors
Deep Learning in Attosecond Metrology.
Optics Express 30.9 (Apr. 2022). Editor's Pick. DOI.
MCML Authors
STEP: Segmenting and Tracking Every Pixel.
Track on Datasets and Benchmarks at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Virtual, Dec 06-14, 2021. PDF.
MCML Authors
Explicit pairwise factorized graph neural network for semi-supervised node classification.
Conference on Uncertainty in Artificial Intelligence (UAI 2021). Virtual, Jul 27-29, 2021. PDF.
MCML Authors
Variational Data Assimilation with a Learned Inverse Observation Operator.
38th International Conference on Machine Learning (ICML 2021). Virtual, Jul 18-24, 2021. URL.
MCML Authors
Post-hoc Uncertainty Calibration for Domain Drift Scenarios.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021). Virtual, Jun 19-25, 2021. DOI.
MCML Authors
Rotation-Equivariant Deep Learning for Diffusion MRI (short version).
International Society for Magnetic Resonance in Medicine Annual Meeting (ISMRM 2021). Virtual, May 15-20, 2021. Long version in arXiv. arXiv.
MCML Authors
Speech Synthesis and Control Using Differentiable DSP.
Preprint at arXiv (Oct. 2020). arXiv.
MCML Authors
MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera.
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Virtual, Jun 14-19, 2020. DOI.
MCML Authors
Deep Learning for 2D and 3D Rotatable Data: An Overview of Methods.
Preprint at arXiv (Oct. 2019). arXiv.