The authors curated 273 tasks from 37,463 deduplicated queries across 26,314 real user coding sessions.
Coding agents satisfy individual requirements more often than complete revisions
Across 273 application-editing tasks, the strongest tested endpoint passed 69.9% of criteria on average but completed only 26.4% of tasks.
Chinese Tech
Xilin Wei · Shengyuan Ding · Xinyu Fang · Tiancheng Gu · Yuchi Wang · Zhixun Li · +10 more
Fudan University · Alibaba Token Hub, Alibaba Group · Zhejiang University · The Chinese University of Hong Kong · Shanghai Jiao Tong University
Research Digest··2 min read
Wei and colleagues introduce Co-Design Bench to test whether coding agents can turn real users’ requests, sometimes accompanied by images or other media, into working application revisions.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.