15

May

Teaser image to Optimal convex M-estimation via score matching

Colloquium

Optimal Convex M-Estimation via Score Matching

Richard Samworth, Cambridge University

   15.05.2024

   5:00 pm - 6:30 pm

   LMU Department of Statistics and via zoom

In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields opti- mal asymptotic variance in the downstream estimation of the regression coefficients. Our semiparametric approach targets the best decreasing approximation of the derivative of the log-density of the noise distribution. At the population level, this fitting process is a nonparametric extension of score matching, corresponding to a log-concave projection of the noise distribution with respect to the Fisher divergence. The procedure is com- putationally efficient, and we prove that our procedure attains the minimal asymptotic covariance among all convex M-estimators. As an example of a non-log-concave setting, for Cauchy errors, the optimal convex loss function is Huber-like, and our procedure yields an asymptotic efficiency greater than 0.87 relative to the oracle maximum likelihood estimator of the regression coefficients that uses knowledge of this error distribution; in this sense, we obtain robustness without sacrificing much efficiency.


Related

Link to Practical Causal Reasoning as a Means for Ethical ML

Colloquium  •  25.06.2025  •  LMU Department of Statistics and via zoom

Practical Causal Reasoning as a Means for Ethical ML

25.06.25, 4:15-5:45 pm: Isabel Valera, Uni Saarbrücken explores fairness in ML and introduces DeCaFlow, a causal model for counterfactuals.


Link to Veridical Data Science and PCS Uncertainty
Quantification

Colloquium  •  11.06.2025  •  LMU Department of Statistics and via zoom

Veridical Data Science and PCS Uncertainty Quantification

11.06.25, 4:15-5:45 pm: Bin Yu, UC Berkeley on how PCS improves AI reliability by tackling hidden uncertainty in data science decisions.