Challenges in Resampling-Based Performance Estimation
MCML Authors
Hannah Schulz-Kümpel
Anne-Laure Boulesteix
Prof. Dr.
Principal Investigator
Roman Hornung
Dr.
Abstract
Hannah Schulz-Kümpel
Anne-Laure Boulesteix
Prof. Dr.
Principal Investigator
Roman Hornung
Dr.
Abstract
As the application of machine learning methods continues to gain relevance in the field of official statistics, the accurate estimation of model performance becomes increasingly crucial. Resampling methods, such as cross-validation and bootstrap, offer powerful tools for estimating the generalization error of predictive models. This chapter explores the concept of generalization error and delves into the intricate challenges associated with conducting inference on it.Building upon the foundational insights, we provide a comprehensive survey of existing methodologies in the literature, offering assessments and recommendations for both uncertainty estimation via confidence intervals and the derivation of point estimates within the context of nonstandard data structures, in particular, non-i.i.d. data.
article SBF+25a
Foundations and Advances of Machine Learning in Official Statistics
Dec. 2025.Authors
H. Schulz-Kümpel • A.-L. Boulesteix • S. Fischer • R. HornungLinks
DOIResearch Area
BibTeXKey: SBF+25a