Code & software

Code generation, coding agents, software engineering with AI, program synthesis

77 articles · page 1 of 2

Code & software

Much of LLM-written database code is safe only by accident

Mahmud, Rahman, and Shahriyar show that more than half of the database code generated by ten language models across 24 backend tasks is safe only because Python's default transaction mode silently provides a guarantee the code never requests. By running the same crash test in a less forgiving autocommit mode, the authors expose code that works in one environment but breaks in another, revealing safety that belongs to the library, not the code.

Code & software

A two-agent system finds bugs better than one by splitting search and inspection

Fang et al. propose NINJA, a hierarchical architecture for issue localization that uses a navigator for file-level search and separate inspectors for function-level analysis in independent contexts. This design prevents context growth from hindering later decisions and allows deeper inspection. After reinforcement learning fine-tuning, NINJA with a 30B-parameter open-source model surpasses the closed-source Claude-Haiku-4.5 model.

Code & software

Robot coding agents learn faster with smarter feedback, not more data

The authors systematically tested how AI coding agents improve robot manipulation skills by varying models, skill libraries, feedback types, and training curricula across 631 experiment configurations and 28,000 simulation rollouts. They found that the main bottleneck is not access to information but the agent's ability to reason about geometry and spatial relationships, and that finer-grained feedback does not always help.

Code & software

Agentic framework formally verifies GPU kernels against PyTorch references

LARK is an agentic framework that automatically and formally verifies the equivalence of GPU kernels (written in languages like CUDA, PTX, or HIP) against their PyTorch reference implementations. It uses the Lean 4 proof assistant to check the kernel's functional and memory behavior under exact arithmetic, shifting the trust boundary: humans only need to verify operator-level semantics once per target language, not every generated kernel. The authors demonstrate that LARK can verify the majority of kernels from benchmarks and production code, including identifying a previously unknown data race bug.

Code & software

Reusing past attempts to steer data collection trains better coding agents

Training data for coding agents is usually gathered by sampling many independent attempts per repository issue and keeping the ones that pass tests. Ziming Li, Lu Yu and colleagues at the University of Connecticut and Ant Group argue that later samples become redundant, and that earlier attempts contain information about where extra sampling is worthwhile. Their collector, SWE-Trail, uses that history to decide when to branch and compare candidate actions, while the agent itself never sees the history, and the resulting trajectories train stronger models than independent sampling does.

Code & software

Malicious project files can push code agents toward vulnerabilities, often without developer awareness

The authors tested two coding agents on 36 secure coding tasks with and without malicious project memory files. They found that the files changed the rate of secure completions by up to 8 percentage points but with no consistent pattern. Crucially, agents differed sharply in whether they told the developer about the file: Claude Code often cited it, while OpenCode almost never mentioned it in the final message.

Code & software

AI translates old codebases into safer Rust at two-thirds lower cost

The authors present SpeC2Rust, a three-stage pipeline that translates entire C repositories into memory-safe Rust. Across eight GitHub C projects, it produces Rust code that compiles and passes the original end-to-end tests with no raw-pointer declarations, no raw-pointer dereferences, and no unsafe lines. Compared with the SmartC2Rust baseline on five shared projects, it reduces monetary cost by 66.1% and input-token usage by 84.7%.

Code & software

Token-level evidence pinpoints AI-written lines and blocks inside mixed code files

Xu, Wang, Liu and He propose a detection framework that keeps evidence at the token level, using two low-rank adapters on a frozen code model, and aggregates that evidence into line, block, and file predictions. On an internal test set of 11,773 mixed-authorship files it reaches 72.4 macro F1 at a 1% false-positive rate. On the external HybridCodeAuthorship benchmark it reaches 62.5 line-level F1 with zero adaptation, exceeding the 56 reported as the best line-level result.

Code & software

LLM code optimization evaluation must check transformation faithfulness, not just speedup

Lu et al. introduce PatternBench, a benchmark of seven compiler-transcendent optimization categories designed to evaluate whether LLMs actually perform the intended transformation, not just produce faster code. By combining speedup measurement with AST-level faithfulness analysis, they show that speedup alone overestimates optimization capability. Their experiments reveal that optimization faithfulness strongly predicts realized performance gains and that fine-tuning improves both faithfulness and effectiveness.

Code & software

Only 5.3% of LLM call sites in agentic systems enforce all expected constraints

The authors present a systematic empirical study of constraints around LLM call sites in open-source agentic systems. Using a multi-agent pipeline validated by 17,428 human judgments, they derive expected constraints from functional specifications and compare them to implemented checks. They find that while 82% of call sites have pre- and post-call checks, only 5.3% satisfy their entire contract, and 63% of expected constraints are not enforced.

Code & software

Security skills guide coding agents to only 11% fully correct vulnerability detection

Xue et al. systematically evaluate 37 public security skills for coding agents (Claude Code with four LLMs) on 100 real Linux kernel vulnerabilities. Across 14,800 runs, only 11.1% of detections were fully correct (vulnerable function, statement, and class). The authors propose SKILL-SIF, a self-improving framework that iteratively refines skills from missed vulnerabilities, improving fully correct detection from 2% to 16% on unseen vulnerabilities.

Code & software

Coding agents improve more when their model and harness co-evolve

Wei et al. introduce Self-HPO, a training framework in which a coding model examines its failed attempts, proposes changes to its own execution scaffold, and then learns under the selected scaffold. Across three software-engineering benchmarks, this alternating process raised the model’s average task-resolution rate from 57.8% to 64.8%, outperforming approaches that optimized only the model or only the harness.

Code & software

LLMs capture only a fraction of human diversity in code generation tasks

Jangra and Muresan introduce DivAlgo, a benchmark of algorithmically diverse human solutions to programming problems, and evaluate 24 large language models (LLMs) for code generation. They find that unconditioned model-generated code captures only about 31% of the algorithmic diversity found in human solutions, and that even the best diverse generation strategies reach only 70.5% of that diversity. The paper concludes that overexposure to model-generated code could narrow programmers' algorithmic vocabulary.

Code & software

8B model learns to construct and refine GPU optimization scripts, beating larger LLMs

The authors introduce a training method that teaches an 8B language model to generate complete GPU optimization scripts and improve existing ones. Trained on search-derived examples from TVM MetaSchedule, the model reaches 91.27% correctness for construction and 92.01% for refinement, and produces faster kernels than six frontier LLMs with hundreds of billions to trillions of parameters.

Code & software

Scaling executable training data for software engineering agents with progressive construction and elastic streaming

The authors present SWE-Stream, a system that scales the construction of executable training data for software engineering (SWE) agents by progressively validating and refining task candidates from development histories. Using an elastic streaming scheduler, it coordinates construction stages asynchronously under shared resource constraints. From 203,905 pull-request candidates across 10,728 repositories and six programming languages, the system produced 80,874 validated tasks and 118,688 successful trajectories; fine-tuning Qwen3-30B-A3B-Thinking-2507 on these data achieved resolve rates of 69.0%, 49.3%, and 45.42% on SWE-bench Verified, Multilingual, and Pro, respectively.

Code & software

Recovered design docs guide agents to rebuild original module structure, not test results.

The authors present a framework that recovers aligned stakeholder intent, requirements and architecture from a compact repository representation (a static dependency graph and key contextual files), then release artefacts for 10,764 repositories in four languages. To test whether the artefacts carry design, a coding agent rebuilt 29 Commit0 Python libraries from the intent alone or with the recovered artefacts; the artefacts significantly improved module-level structural similarity (F1 from 0.54 to 0.63) without significantly changing the held-out pass rate (33.7% vs 35.8%).

Code & software

Execution-verified feedback improves LLM harness optimization for SQL, Cypher, and Python generation

The authors introduce HERO, a two-stage framework for optimizing the harness (prompts and scaffolding) around LLM program generation. The offline stage learns reusable prompt updates from execution-verified successes and failures, while test-time refinement treats each generated program as an instance-specific harness. Across four SQL, Cypher, and Python benchmarks, HERO improves pass@1 by 7.6 points over the initial prompt, with a further 2.0 points from test-time refinement.

Code & software

Adaptive test weighting gives software agents denser, more useful rewards

Xu and colleagues show that raw test pass-rate is a poorly calibrated reinforcement-learning reward for software engineering agents because test suites overrepresent some functions, difficulties and regression checks. Their CodeLift method dynamically reweights behaviorally distinct test groups and evaluates intermediate repository snapshots, outperforming binary and pass-rate rewards across two SWE benchmarks.

Code & software

Framework scales verifiable training environments for coding agents that build programs from scratch

The authors introduce ProgramGym, a framework for training coding agents that reproduce a program's observable behavior from documentation and a reference executable, without source code. Its environment pipeline builds verifiable black-box testing tasks across C, C++, Go and Rust, and its agent workflow coordinates analysis, implementation and review over long horizons. Fine-tuning on the resulting trajectories lifts Qwen3.8-27B from 34.87% to 42.84% on ProgramBench.

Code & software

Learning the refinement loop improves web generation beyond final pages

The authors introduce WebLoop, a framework that trains a generator, critic, and refiner as roles of a shared policy for functional web generation. The critic learns to diagnose requirements without execution traces, receiving credit from downstream improvement. On WebRise and WebGen-Bench, WebLoop with Qwen3.5-9B improves the base model by 11.3 and 15.4 points respectively, and gains persist at 27B scale and to multimodal inputs.