Home | Publications | HLZ+26

Spatial-X Fusion for Multi-Source Satellite Imageries

MCML Authors

Jiang He

Dr.

→ Group Xiaoxiang Zhu
Data Science in Earth Observation

Xiaoxiang Zhu

Prof. Dr.

Principal Investigator

Data Science in Earth Observation

Abstract

Multi-source remote sensing data can highlight different types of information based on user needs, resulting in large volumes of data and significant challenges. Hardware and environmental constraints create mutual dependencies between information types, particularly between spatial data and other types, limiting the development of high-precision applications. Traditional methods are task-specific, leading to many algorithms without a unified solution, which greatly increases the computational and deployment costs of image fusion. In this paper, we summarize four remote sensing fusion tasks, including pan-sharpening, hyperspectral-multispectral fusion, spatio-temporal fusion, and polarimetric SAR fusion. By defining the spectral, temporal, and polarimetric information, as X, we propose the concept of generalized spatial-channel fusion, referred to as Spatial-X fusion. Then, we design an end-to-end network SpaXFus, a generalized spatial-channel fusion framework through a model-driven unfolding approach that exploits spatial-X intrinsic interactions to capture internal dependencies and self-interactions. Comprehensive experimental results demonstrate the superiority of SpaXFus, e.g., SpaXFus can achieve four remote sensing image fusion tasks with superior performance (across all fusion tasks, spectral distortion decreases by 25.48 %, while spatial details improve by 7.5 %) and shows huge improvements across multiple types of downstream applications, including vegetation index generation, fine-grained image classification, change detection, and SAR vegetation extraction.

article HLZ+26