Home | Events

10

Jul

Teaser image to Variational Learning for Large Deep Networks

 

Colloquium

Variational Learning for Large Deep Networks

Thomas Möllenhoff, RIKEN, Tokyo

   10.07.2024

   3:15 pm - 4:45 pm

   LMU Department of Statistics and via zoom

Thomas Möllenhoff presents extensive evidence against the common belief that variational Bayesian learning is ineffective for large neural networks.

First, he shows that a recent deep learning method called sharpness-aware minimization (SAM) solves an optimal convex relaxation of the variational Bayesian objective.

Then, he demonstrates that a direct optimization of the variational objective with an Improved Variational Online Newton method (IVON) can consistently match or outperforms Adam for training large networks such as GPT-2 and ResNets from scratch. IVON’s computational costs are nearly identical to Adam but its predictive uncertainty is better.

He shows several new use cases of variational learning where he improves fine-tuning and model merging in Large Language Models, accurately predict generalization error, and faithfully estimate sensitivity to data.

Organized by:

Department of Statistics LMU Munich

Related

Link to Active learning-assisted neutron spectroscopy with log-Gaussian processes

 

Colloquium  •  08.11.2023  •  LMU Department of Statistics and via zoom

Active learning-assisted neutron spectroscopy with log-Gaussian processes

Join the lecture with Mario Teixeira Parente, Department of Statistics, LMU Munich.

Link to Applied Causal Inference with Surrogate Representation

©geralt - pixabay.com

AI Keynote Series  •  20.07.2023  •  LMU Institute of AI in Management via zoom

Applied Causal Inference with Surrogate Representation

Join the presentation of Lu Cheng, University of Illinois, Chicago.

Link to Assessing goodness of fit for network models

 

Colloquium  •  15.02.2023  •  LMU Department of Statistics and via zoom

Assessing goodness of fit for network models

Join the presentation of Gesine Reinert, University of Oxford, on network models.

Link to Auditing Fairness under Unobserved Confounding

©jittawit.21 - stock.adobe.com

AI Keynote Series  •  08.08.2024  •  Online via Zoom

Auditing Fairness under Unobserved Confounding

Join the lecture with Michael Oberst from Johns Hopkins University.

Link to AutoML for tabular datasets and tabular datasets for AutoML

 

Colloquium  •  06.12.2023  •  LMU Department of Statistics and via zoom

AutoML for tabular datasets and tabular datasets for AutoML

Join the lecture with our PI Matthias Feurer, LMU Munich, giving a short introduction into AutoML and an overview of the AutoML research at his chair.