Massively Multilingual Language Modeling and Adaptation
MCML Authors
Peiqin Lin
Dr.
* Former Member
Abstract
Peiqin Lin
Dr.
* Former Member
Abstract
This dissertation advances language modeling and task adaptation for low-resource languages by developing large-scale multilingual models and improving cross-lingual transfer. It introduces the Glot500, MaLA500, and EMMA500 frameworks to expand pretrained models to hundreds of languages and achieve state-of-the-art multilingual performance. In addition, it proposes new methods for exploiting parallel corpora and measuring language similarity, and presents XAMPLER for effective cross-lingual in-context learning, significantly improving performance in low-resource settings. (Shortened.)
BibTeXKey: Lin25