Home | Publications | LNL23

From Classification to Generation: Insights Into Crosslingual Retrieval Augmented ICL

MCML Authors

Ercong Nie

→ Group Hinrich Schütze
Computational Linguistics

Sheng Liang

* Former Member

→ Group Hinrich Schütze
Computational Linguistics

Abstract

The remarkable ability of Large Language Models (LLMs) to understand and follow instructions has sometimes been limited by their in-context learning (ICL) performance in low-resource languages. To address this, we introduce a novel approach that leverages cross-lingual retrieval-augmented in-context learning (CREA-ICL). By extracting semantically similar prompts from high-resource languages, we aim to bolster the zero-shot performance of multilingual pretrained language models (MPLMs) across diverse tasks. Though our approach yields steady improvements in classification tasks, it faces challenges in generation tasks, with Bangla serving as a key case study. Our evaluation offers insights into the performance dynamics of retrieval-augmented in-context learning across both classification and generation domains.

inproceedings

Instruction Tuning and Instruction Following @NeurIPS 2023

Workshop Instruction Tuning and Instruction Following at the 37th Conference on Neural Information Processing Systems. New Orleans, LA, USA, Dec 10-16, 2023.