The authors assembled 100 enhancement tasks from 29 large software systems spanning five domains and multiple programming languages.
Coding agents struggle to turn software proposals into working changes
Across 100 large repository tasks, the best of 28 agents completed only 14 percent without additional implementation guidance.
Top University
Yun Peng · Zihan Wu · Zeyang Zhuang · Xin Zhou · Rui Shu · Xu Han · +3 more
Fudan University · City University of Hong Kong · Chinese University of Hong Kong · Singapore Management University · Independent Researcher
Research Digest··3 min read
Peng et al.
Why this paper
From Fudan University and 7 others · Released data
In one line
Coding agents resolve at most 14% of long-horizon enhancement proposal tasks on large software systems, and incomplete code localization is a major bottleneck.
What it released
Data
What we could check
- ·No code link found
- ·No weights link found
- ✓Dataset link in the paper (huggingface.co)
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§