STEEREVAL: Inference-Time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics

MCML Authors

Felicia Körner

→ Group Barbara Plank
 AI and Computational Linguistics

Barbara Plank

Prof. Dr.

Core PI

AI and Computational Linguistics

Abstract

An empirical bottleneck for Natural Language Generation tasks such as summarization is the lack of accurate and robust evaluation metrics for many languages, which hinders progress. While Large Language Models (LLMs) are increasingly adopted as evaluators with the advent of LLM-as-a-Judge approaches, their agreement with human judgments remains low across many languages. We take inspiration from recent studies suggesting that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance during text generation. Motivated by this hypothesis, we ask whether steering the activations of neural metrics with respect to an English pivot can improve their correlation with human judgments in multilingual summarization evaluation. We experiment with encoder- and decoder-based metrics across five languages. We find that activation steering is effective for neural metrics, and that a single, language-independent steering vector improves metric–human agreement. Steering also results in improvements when applied to COMET, a small encoder-based metric, showing notable human agreement despite its small scale.

inproceedings CSK+26

---

EMNLP 2026

Conference on Empirical Methods in Natural Language Processing. Budapest, Hungary, Oct 24-29, 2026. To be published. Preprint available.

---

Authors

S. Casola • R. S.-E. Shim • F. Körner • Y. Mao • B. Plank

---

Links

URL

---

Research Area

 B2 | Natural Language Processing