Home | Publications | KZT+26

CooperBench: Why Coding Agents Cannot Be Your Teammates Yet

MCML Authors

Abstract

Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. Similarly, as AI agents increasingly collaborate on complex work, they must develop coordination capabilities to function as effective teammates. We hypothesize that current agents lack these capabilities. To test it, we introduce CooperBench, a benchmark of 652 collaborative coding tasks spanning 12 libraries and 4 languages. Each task assigns agents independently implementable features that may conflict without coordination. Tasks are grounded in real open-source repositories with expert-written tests, which makes the cooperation outcomes fully verifiable. Evaluating SOTA coding agents, we observe the curse of coordination: at least 30% average drop in success when agents work together versus a single agent completing both tasks, especially when the tasks are not extremely easy or hard. Critically, even on task pairs where each agent can individually solve its assigned feature, the best models still fail to coordinate roughly half the time, a conditional failure rate that cannot be explained by coding difficulty alone. We also find that human developer pairs solve 9 out of 10 tasks under the same coordination constraints that agents handle at only 3 out of 10, establishing that the overhead of splitting work is not the root cause. We identify three failure modes: (1) communication channels become jammed with vague, ill-timed, and inaccurate messages; (2) even with good communication, agents deviate from their commitments and hold incorrect expectations about others; and (3) agents have troubles using version control tools to perform real-time coordination. Beyond an open-source benchmark, this work contributes a novel understanding of what agents need to learn to become effective teammates.

inproceedings KZT+26

COLM 2026

Conference on Language Modeling. San Francisco, CA, USA, Oct 06-09, 2026.

Authors

A. Khatua • H. Zhu • P. Tran • A. Prabhudesai • F. Sadrieh • J. K. Lieberwirth • X. Yu • Y. Fu • M. J. Ryan • J. Pei • D. Yang

Links

URL

Research Area

 B2 | Natural Language Processing

BibTeXKey: KZT+26

#p-schuetze
Back to Top