This paper introduces WideSWE, a benchmark of 120 real-world tasks that require coordinated changes across multiple code repositories. The authors find that current , including the best configuration (Codex CLI with GPT-5.6-sol), achieve only 42.50% success. Agents often miss necessary changes, leave work unfinished, or make incorrect modifications. Independent execution on one repository at a time recovers omitted work but does not improve overall success compared to joint execution of all repositories.
Coding agents fail at coordinated multi-repository programming tasks
New benchmark WideSWE shows top agent solves only 42.5% of real-world cross-repository tasks, with failures in scope identification and incomplete changes.
Independent
Baoyi Wang · Xingliang Wang · Jinyang Wu · Keming Wu · Chen Zhi · Jianwei Yin
Research Digest··2 min read
In one line
Coding agents succeed on only 42.50% of cross-repository software tasks.
Before you read this
Three ideas this paper builds onagentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real316 papers →coding agentAn AI-powered software system that autonomously writes, modifies, or extends source code based on high-level instructions.178 papers →repository-level benchmarkAn evaluation suite that measures agent performance on tasks spanning entire codebases, from construction to repair and governance.48 papers →
Read it as
The authors introduce WideSWE, a benchmark of 120 real-world software engineering tasks that require coordinated changes across multiple repositories.
The authors constructed WideSWE by mining merged pull requests across 103 software ecosystems from the top 200 GitHub organizations.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.