MICDrop: Masking Image and Depth Features via Complementary Dropout for Domain-Adaptive Semantic Segmentation
MCML Authors
Mark Weber
* Former Member
Daniel Cremers
Prof. Dr.
Director
Abstract
Mark Weber
* Former Member
Daniel Cremers
Prof. Dr.
Director
Abstract
Unsupervised Domain Adaptation (UDA) is the task of bridging the domain gap between a labeled source domain, e.g., synthetic data, and an unlabeled target domain. We observe that current UDA methods show inferior results on fine structures and tend to oversegment objects with ambiguous appearance. To address these shortcomings, we propose to leverage geometric information, i.e., depth predictions, as depth discontinuities often coincide with segmentation boundaries. We show that naively incorporating depth into current UDA methods does not fully exploit the potential of this complementary information. To this end, we present MICDrop, which learns a joint feature representation by masking image encoder features while inversely masking depth encoder features. With this simple yet effective complementary masking strategy, we enforce the use of both modalities when learning the joint feature representation. To aid this process, we propose a feature fusion module to improve both global as well as local information sharing while being robust to errors in the depth predictions. We show that our method can be plugged into various recent UDA methods and consistently improve results across standard UDA benchmarks, obtaining new state-of-the-art performances.
inproceedings YHW+24a
ECCV 2024
18th European Conference on Computer Vision. Milano, Italy, Sep 29-Oct 04, 2024.
Authors
L. Yang • L. Hoyer • M. Weber • T. Fischer • D. Dai • L. Leal-Taixé • D. Cremers • M. Pollefeys • L. Van GoolLinks
DOI GitHubIn Collaboration
Microsoft
NVIDIA
Research Area
BibTeXKey: YHW+24a