Code & software

Code generation, coding agents, software engineering with AI, program synthesis

23 articles

All Agents Language models Reinforcement learning Vision Generative media Speech & audio Robotics & embodied AI Safety & security Evaluation & benchmarks Efficiency & systems Code & software AI for science & health Theory

Viewgrid list compact

Code & software

Training models on runtime program-state reasoning boosts automated software engineering

The authors introduce two complementary program-state reasoning tasks—buggy input-output reasoning and precondition-postcondition reasoning—and incorporate them into a staged post-training pipeline to build Comet-9B, a 9B-parameter language model. They find that adding both tasks to supervised fine-tuning on issue resolution improves success rates by 7.25 percentage points on SWE-bench Pro and 9.70 points on SWT-Bench Verified, with further gains from sequential reinforcement learning. Despite its small size, Comet-9B achieves scores comparable to reported results from GPT-5.2 and GPT-4o-based agents.

3 Oct·3 min

Code & software

Visual feedback boosts software engineering agent success rates

The authors introduce CUA-SWE, a benchmark and evaluation pipeline for software engineering with computer use. They find that frontier agents which can view and interact with application interfaces complete tasks more successfully than agents limited to code editing, especially in web and game domains.

3 Oct·2 min

Code & software

Training recipe boosts security in AI-generated code by improving planning and testing

The authors developed SECUREVIBE, a training recipe that explicitly targets planning and testing for code security. They found that insecure agents are less than half as likely to conduct effective security planning and testing. SECUREVIBE improves security pass@1 by 6.9 points on BaxBench and functionality pass@1 by 13.6 points on SusVibes.

1 Oct·2 min

Code & software

New benchmark tests AI's ability to spot invalid code reviews

The authors introduce CRJudgeBench, a benchmark of 1199 instances built from real pull requests and expert-verified perturbations, assessing whether AI agents can distinguish technically valid code review comments from plausible but invalid ones. They also present Sentinel, a repository-grounded agentic judge trained via iterative action-level learning, which achieves 76.60% accuracy on the test set, substantially outperforming base models and showing that general-purpose LLMs struggle with this task.

30 Sept·2 min

Code & software

Code-linked evidence lets AI software claims expire when dependencies change

Tiwari, Vass and Singh present Assay, a Python system that combines repository indexing with an evidence ledger for AI-assisted software delivery. Each claim about code is tied to a Merkle hash of the relevant dependency cone, meaning the module and everything it depends on, so changes invalidate precisely the claims that may no longer hold. Across five public repositories, this approach reduced unnecessary re-verification while a model-free merge gate blocked all nine scripted adversarial behaviors.

30 Sept·3 min

Code & software

Agents learn software repair rules that transfer across repositories

Guo et al. present XRepoSkill, a method for extracting reusable software engineering guidance from agents’ prior repair attempts without retraining the underlying language model. Across two held-out benchmarks and three backbone models, it achieved the highest issue resolution rate among the evaluated skill-learning methods, with its largest reported gain on the long-horizon DeepSWE benchmark.

30 Sept·2 min

Code & software

Merged coding-agent performance fixes often miss their claimed gains

Qi and colleagues examined how maintainers handle performance fixes submitted by coding agents, then reran a subset of accepted and rejected patches. Although 57% of closed fixes were merged, only 18 of 30 re-executed merged fixes met the authors’ delivery criterion, showing that acceptance is not reliable evidence of a real performance improvement.

30 Sept·2 min

Code & software

Training LLMs to write code boosts medical calculation accuracy by 20-30 points

Ming et al. introduce MedCode, a framework that fine-tunes LLMs to generate embedded executable code for medical calculations rather than producing numeric answers directly. On the MedCalc benchmark, LLaMA3-8B, Qwen2.5-7B, and Mistral-7B show absolute accuracy gains of 20-30 percentage points compared to direct text generation, with the best model reaching 94% accuracy.

30 Sept·3 min

Code & software

Query plans make text-to-SQL portable across database dialects

The authors show that text-to-SQL systems trained on a single SQL dialect (e.g., SQLite) suffer substantial accuracy drops when deployed on others. They propose generating a dialect-agnostic query plan, which a deterministic compiler translates to any backend SQL. Across 13 models from 3B to frontier scale, this restores cross-dialect portability nearly uniformly with minimal or no accuracy loss.

30 Sept·2 min

Code & software

Adoption-aware memory learning boosts LLM agent NPU kernel synthesis

Fan et al. present SAGE, a self-improving agent that uses adoption-aware credit assignment and selective consolidation to synthesize high-performance NPU kernels. On NPUKernelBench, SAGE achieves a 95.5% execution rate and 86.9% of solved operators outperform torch_npu, with a 43.99x speedup on sparse flash attention.

30 Sept·2 min

Code & software

Semantic constraints can prune invalid code during language model decoding

Kronlund-Drouault introduces semantic grammar specifications, which add contextual program constraints to grammar-guided language model decoding. Tests against production compilers found no false rejections across prefixes of 65 valid programs, while generation experiments showed nonnegative observed gains over syntax-only decoding for every tested model-language pair.

29 Sept·2 min

Code & software

LLM-discovered search operators adapt destruction and repair to search state

Ke et al. introduce Stackelberg Program Optimization, an LLM-based method for discovering paired destroy and repair programs for combinatorial optimization. On traveling salesperson and capacitated vehicle routing problems, the authors report stronger results than handcrafted, test-time search, and program-discovery baselines, including on instances larger than those used during discovery.

29 Sept·2 min

Code & software

One demonstration can bootstrap reusable robot manipulation programs

Liu et al. present RAPID, a system that converts a single visual human demonstration into a robot manipulation program. Rather than copying demonstrated motions, it represents object-level effects and spatial relationships, allowing the resulting program to adapt to changes in objects and scenes.

28 Sept·2 min

Code & software

Reusable skills improve smart-contract audits, but model choice dominates

The authors collected smart-contract auditing skills used in the wild and evaluated them on EVMBench while analyzing how they changed agent execution. Codex with GPT-5.5 produced the largest gains, but inconsistent skill triggering emerged as a central obstacle to reliable improvement.

27 Sept·2 min

Code & software

New benchmark tests LLMs on repository-level reasoning about code execution

Taherkhani et al. introduce SWE-Flux, a benchmark for evaluating large language models' ability to reason about dynamic runtime behavior at the repository level. The authors show that state-of-the-art LLMs achieve at most 37% accuracy, indicating significant room for improvement.

25 Sept·2 min

Code & software

AI coding agents still miss production inference failures

The authors built 53 tasks from recent production changes to the SGLang inference-serving system and evaluated 11 models across 31 model-effort configurations. The best configuration reached 75% mean pass@1, but on tasks with end-to-end coverage, roughly one-third of patches passing all other checks still failed full serving tests.

24 Sept·2 min

Code & software

Coding agents retrieve better by targeting missing evidence sets

Feng et al. frame retrieval for coding agents as finding the smallest set of evidence that complements what an agent has already seen. On 500 agent states from 45 repositories, their MSS-Complement method recovered all annotated evidence requirements more often than embedding-based retrieval and similarity-only ranking.

19 Sept·2 min

Code & software

Program repair improves when memories match repository needs and repair stages

Luo and colleagues developed an adaptive memory-retrieval framework for large-language-model-based repository repair. On SWE-Bench-Lite and SWE-Bench-Verified, the authors report better performance for repositories with sparse histories, less noisy retrieval, and stronger refinement of initially unsuccessful patches.

19 Sept·2 min

Code & software

Embedded software agents need interaction contracts and continuous assurance

Yu, Fang, and Chen propose Agent-Integrated Software, a pattern combining a conventional application core, direct user interaction, and a built-in agent. Their transition-system framework connects user-facing task semantics to application execution, while interaction contracts and continuous assurance specify when delegated actions and outcomes remain valid.

12 Sept·2 min

Code & software

Literate programming links prose, code and symbols for coding agents

Burke presents a working literate-programming environment designed for collaboration between human programmers and language-model agents. It co-locates code with relevant prose and structured data while exposing symbol-like name relationships that agents can search and navigate.

27 Aug·2 min

Code & software

Coding agents extend LLM serving simulators with greater fidelity and speed

Kim et al. built Simthesizer, a framework in which a harnessed coding agent modifies a shared simulator of LLM serving workflows rather than constructing a separate simulator for each new mechanism. The authors report 2.51% average throughput error against a vLLM-based system and simulation speeds up to 284.96 times those of competing simulators.

27 Aug·2 min

Code & software

Switching Models Mid-Task Imposes a Cost and Quality Penalty

Ganz et al. tested coding agents that switched between lower-cost, lower-capability models and higher-cost, higher-capability models during long-running tasks. They find a “handoff tax”: giving a stronger model the weaker model’s full trajectory recovers less than half the quality gap between them while adding substantial cost.

27 Aug·2 min

Code & software

Post-training can replace prompt-time schema injection for enterprise coding agents

The authors present CRAFT, a two-stage post-training recipe that enables enterprise coding agents to learn schema and tool-use behavior without exhaustive prompt-time schema injection. They show that CRAFT improves composite Agent Score by +9.6 percentage points, reduces input tokens by ~9x, and cuts schema-discovery loops by up to 5x compared to a schema-stuffed baseline.

24 June·2 min