The paper introduces AgentWorld, a of 100 human-annotated tasks (plus 100 augmented variants) for evaluating long-horizon multi-agent collaboration. Tasks require 3-20 agents with asymmetric roles to coordinate over 25-55 rounds in a blackbox MMORPG sandbox where agents cannot see each other's internal states. The benchmark abstracts away low-level controls through 13 high-level API tools, focusing on collaboration rather than action proficiency. A new metric, Causal Collaboration Effectiveness (CCE), traces causal dependencies between agent actions to measure what fraction of efforts actually contributed to task success. Experiments with four frontier models (Gemini 3 Flash, Claude Haiku 4.5, GPT-5 Mini, DeepSeek R1-70B) found the best model achieving only 52.0% task success and a CCE of 0.320, meaning less than a third of actions were causally impactful. Systematic failure modes included communication breakdowns, role confusion, and inability to maintain shared plans. This is interesting because it highlights that current LLM agents struggle with sustained, complex collaboration even when abstracted from low-level control.
Current LLM agents struggle to sustain long-horizon team collaboration
AgentWorld tests teams of 3 to 20 agents over more than 50 interaction rounds, with the best evaluated model completing only 52.0 percent of tasks.
Top University
OpenAgents · Columbia University · University of Pennsylvania · Seoul National University · Penn State University
Research Digest··2 min read
In one line
AgentWorld shows that multi-agent LLMs achieve at most 52% success on long-horizon collaboration tasks with 25-55 rounds.
Before you read this
Three ideas this paper builds onmulti-agent coordinationThe collaborative operation of multiple AI agents to solve tasks more effectively or with greater capability.91 papers →large language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.329 papers →benchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.366 papers →
Read it as
The authors introduce AgentWorld, an MMORPG-based benchmark designed to isolate collaboration among independently acting LLM agents with asymmetric roles and abilities.
The authors built 100 human-annotated collaborative tasks and 100 augmented variants in an MMORPG sandbox.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.