Approximating Positive Homogeneous Functions With Scale Invariant Neural Networks
MCML Authors
Reinhard Heckel
Prof. Dr.
Principal Investigator
Felix Krahmer
Prof. Dr.
Principal Investigator
Abstract
Reinhard Heckel
Prof. Dr.
Principal Investigator
Felix Krahmer
Prof. Dr.
Principal Investigator
Abstract
We investigate the approximation of positive homogeneous functions, i.e., functions satisfying for all , with neural networks. Extending previous work, we establish new results explaining under which conditions such functions can be approximated with neural networks. As a key application for this, we analyze to what extent it is possible to solve linear inverse problems with networks. Due to the scaling invariance arising from the linearity, an optimal reconstruction function for such a problem is positive homogeneous. In a network, this condition translates to considering networks without bias terms. For the recovery of sparse vectors from few linear measurements, our results imply that networks with two hidden layers allow approximate recovery with arbitrary precision and arbitrary sparsity level in a stable way. In contrast, we also show that with only one hidden layer such networks cannot even recover 1-sparse vectors, not even approximately, and regardless of the width of the network. These findings even apply to a wider class of recovery problems including low-rank matrix recovery and phase retrieval. Our results also shed some light on the seeming contradiction between previous works showing that neural networks for inverse problems typically have very large Lipschitz constants, but still perform very well also for adversarial noise. Namely, the error bounds in our expressivity results include a combination of a small constant term and a term that is linear in the noise level, indicating that robustness issues may occur only for very small noise levels.
article
Journal of Approximation Theory
311.106177. Nov. 2025.Authors
S. Bamberger • R. Heckel • F. KrahmerLinks
DOIResearch Area
BibTeXKey: BHK25