Crosslingual On-Policy Self-Distillation for Multilingual Reasoning

MCML Authors

Yihong Liu

Dr.

→ Group Hinrich Schütze
 Computational Linguistics

Raoyuan Zhao

→ Group Michael Hedderich
 AI and Computational Linguistics

Michael Hedderich

Dr.

JRG Leader Human-Centered NLP

AI and Computational Linguistics

Hinrich Schütze

Prof. Dr.

Core PI

Computational Linguistics

Abstract

Large language models (LLMs) have achieved remarkable progress in mathematical reasoning, but this ability is not equally accessible across languages. Especially low-resource languages exhibit much lower reasoning performance. To address this, we propose Crosslingual On-Policy Self-Distillation (COPSD), which transfers a model's own high-resource reasoning behavior to low-resource languages. COPSD uses the same model as student and teacher: the student sees only the low-resource problem, while the teacher receives privileged crosslingual context, including the problem translation and reference solution in English. Training minimizes full-distribution token-level divergence on the student's own rollouts, providing dense supervision while avoiding the sparsity and instability of outcome-only reinforcement learning (RL). Experiments on 17 low-resource African languages show that COPSD consistently improves low-resource mathematical reasoning across model sizes and substantially outperforms Group Relative Policy Optimization (GRPO). Further analyses show that COPSD improves answer-format adherence, strengthens test-time scaling, and generalizes to harder multilingual reasoning benchmarks, with especially large gains for lower-resource languages.

misc LZH+26

---

Preprint

May. 2026

---

Authors

Y. Liu • R. Zhao • M. A. Hedderich • H. Schütze

---

Links

arXiv GitHub

---

Research Area

 B2 | Natural Language Processing