Code & software research

The frontier

Training models on runtime program-state reasoning boosts automated software engineering

The authors introduce two complementary program-state reasoning tasks—buggy input-output reasoning and precondition-postcondition reasoning—and incorporate them into a staged post-training pipeline to build Comet-9B, a 9B-parameter language model. They find that adding both tasks to supervised fine-tuning on issue resolution improves success rates by 7.25 percentage points on SWE-bench Pro and 9.70 points on SWT-Bench Verified, with further gains from sequential reinforcement learning. Despite its small size, Comet-9B achieves scores comparable to reported results from GPT-5.2 and GPT-4o-based agents.

Hongwei Li, Spandan Garg, Yufan Huang

Code & software

Visual feedback boosts software engineering agent success rates

The authors introduce CUA-SWE, a benchmark and evaluation pipeline for software engineering with computer use. They find that frontier agents which can view and interact with application interfaces complete tasks more successfully than agents limited to code editing, especially in web and game domains.

today

Evaluation & benchmarks

Coding agents nearly finish repositories but still miss explicit requirements

Yang et al. introduce Zero2Repo, a benchmark that gives coding agents an empty workspace, a product requirements document and an interface contract, then asks them to deliver an entire software repository. On 11 tasks, the strongest evaluated agent completed 10, while failures were usually nearly correct implementations undone by one omission or uncommon rule.

yesterday

Agents

Reinforcement learning helps coding agents finish tasks across benchmarks

Mehta, Ritchie and Chen post-trained Kimi K2.7 Code using reinforcement learning with verifiable rewards, without a supervised warm-up. The resulting model achieved higher pass@1 on every evaluated benchmark, including later-released task sets, unfamiliar agent harnesses and multi-hour software projects.

yesterday

AgentsMSFT ▲0.9%

Coding agents can improve by evolving their own instruction rules

Jiang et al. developed RuleEvolve, a framework that automatically refines the instruction files used to guide AI coding agents. Across two agent frameworks, four backbone language models, and three coding benchmarks, the evolved rules matched or improved functional correctness while often producing shorter code and using fewer generation resources than manual rules and prompt-optimization baselines.

yesterday

Code & software

Training recipe boosts security in AI-generated code by improving planning and testing

The authors developed SECUREVIBE, a training recipe that explicitly targets planning and testing for code security. They found that insecure agents are less than half as likely to conduct effective security planning and testing. SECUREVIBE improves security pass@1 by 6.9 points on BaxBench and functionality pass@1 by 13.6 points on SusVibes.

2 days ago

Agents

Training on self-generated algorithms helps design agents escape performance plateaus

Lu et al. address a recurring problem in evolutionary algorithm-design agents: repeatedly refining solutions in context can stall when the task requires specialized knowledge. Their PCPO method instead trains the model on a curated population of its own solutions, outperforming frozen-model evolutionary approaches across chip-placement cases and producing substantial GPU-kernel speedups.

2 days ago

All Code & software papers →

Every paper read and written up by the research desk from the daily arXiv scan · threads are maintained lines of inquiry with running syntheses