PACE converts enterprise development records into standalone PRDs, then extracts candidate requirements in two independent passes: one from the PRD alone, one from the reference code diff.
Coding agents fully deliver enterprise PRD requirements only 14 percent of the time
The PACE benchmark scores 43 production tasks against 750 expert-audited acceptance criteria and traces incomplete delivery to missing implementation and integration steps.
Chinese Tech
Tianqi Liu · Yandong Guan · Songze Li · Wenkai Wang · Gang Yu · Chengfei Lv
Zhejiang University · Alibaba Taobao & Tmall Group
Research Digest··3 min read
The authors introduce PACE, a benchmark of 43 production enterprise tasks with 750 expert-audited Code Points, which evaluates whether coding agents fulfill complete Product Requirement Documents (PRDs) in mature repositories.
§