NARRA-Gym for Evaluating Interactive Narrative Agents

MCML Authors

Yuchen Ma

→ Group Stefan Feuerriegel
 Artificial Intelligence in Management

Stefan Feuerriegel

Prof. Dr.

Core PI

Artificial Intelligence in Management

Abstract

Interactive narrative tasks require LLMs to sustain a coherent, evolving story while adapting to a user over multiple turns. However, suitable benchmarks for this setting are limited: existing evaluations often focus on static prompts, isolated story generations, or post-hoc ratings, and therefore miss whether models can jointly manage story generation, long-context state and pacing, character simulation, empathic personalization, and story-grounded artifacts. We introduce NARRA-Gym, an executable evaluation environment that turns a sparse emotional seed into a complete interactive story episode and logs the full model-in-the-loop trajectory, including story construction, memory updates, planning, pacing interventions, and optional artifact synthesis. We evaluate nine frontier LLMs using a controlled LLM-as-judge sweep over eight benchmark personas and a human evaluation in which participants rate customized model outputs. Our results show substantial variation across models, personas, and evaluation dimensions: models that produce fluent stories can still fail on robustness, user experience, or resistance-sensitive personalization. These findings suggest that interactive narrative offers a useful benchmark for evaluating long-horizon, user-adaptive LLM behavior beyond isolated story quality.

misc HMY+26

---

Preprint

May. 2026

---

Authors

Y. Huang • Y. Ma • J. Ye • W. Wang • Z. Ling • X. Hu • Y. Hao • Z. Chen • Z. Xu • Y. He • Z. Yuan • Y. Zhou • K. Guo • C. Chen • T. Li • S. Feuerriegel • X. Zhang

---

Links

arXiv GitHub

---

Research Area

 A1 | Statistical Foundations & Explainability