Program reproduction tasks such as ProgramBench ask an agent to rebuild a program's behavior from usage documentation and a reference program, with no source code or network access.
Framework scales verifiable training environments for coding agents that build programs from scratch
ProgramGym's feedback-driven pipeline yields 1,648 tasks across four languages and lifts a 27B model to surpass a flagship system on ProgramBench.
Research Lab
Chengyi Yang · Renxu Bao · Jue Hou · Zongpei Teng · Ziheng Wang · Yujie Lin · +7 more
Xiamen University · Washington University in St. Louis · Institute of Automation, Chinese Academy of Sciences · Unisound
Research Digest··2 min read
The authors introduce ProgramGym, a framework for training coding agents that reproduce a program's observable behavior from documentation and a reference executable, without source code.
Why this paper
From Institute of Automation, Chinese Academy of Sciences and 3 others
In one line
ProgramGym's feedback-driven environment construction and role-isolated agent workflow improve from-scratch program reproduction, lifting Qwen3.8-27B to 55.71% on ProgramBench.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§