Harnesses & orchestration research

The frontier

LLM agent scaffolding systematically finds broken access control vulnerabilities in web apps

Duarte et al. present ABSENTIA, a security scaffolding that directs LLM agents to systematically audit web application backends for broken access control. The agents build a route–code graph and apply invariant falsification to infer intended authorization properties, reporting routes where they are not enforced. On a new benchmark of 30 real-world advisories (BAC-Bench), ABSENTIA recalls 19 vulnerabilities with 17 under strict credit; CodeQL and Semgrep recall none.

André V. Duarte, Aditya Oke, Rui Melo +7

Agents

Evidence-driven harness edits improve frozen GUI agents across benchmarks

Yang et al. introduce an automated method for improving the runtime harness around a GUI agent without changing the underlying model. Across six backbones on OSWorld-Verified, the authors report consistent held-out improvements, including a 12.33-point gain for Qwen3-VL-32B-Instruct. A harness optimized on one benchmark also transferred to WindowsAgentArena, where it improved GPT-5 by 13.87 percentage points.

yesterday

Agents

Agent performance depends on matching models, harnesses and tasks

Li and colleagues evaluate language models together with the software harnesses that expose tools, manage context and return errors. Across 66 configurations and 6,204 scored trajectories, they find no consistently best model or harness: rankings often reverse when either the harness or task changes, while higher spending and vendor-native pairings do not reliably improve results.

yesterday

AgentsMSFT ▲0.9%

Coding agents can improve by evolving their own instruction rules

Jiang et al. developed RuleEvolve, a framework that automatically refines the instruction files used to guide AI coding agents. Across two agent frameworks, four backbone language models, and three coding benchmarks, the evolved rules matched or improved functional correctness while often producing shorter code and using fewer generation resources than manual rules and prompt-optimization baselines.

yesterday

Agents

Adaptive training curricula produce stronger optimized agent harnesses

Park et al. treat scenario selection as part of agent harness optimization rather than as a fixed input to it. Their system, ActiveSaddler, adaptively revisits unresolved failure types while searching for new weaknesses, raising test Pass@1 by 4.4 percentage points on GAIA2 and 7.5 points on Terminal-Bench 2.0 relative to the same optimizer using a predetermined scenario order.

yesterday

Safety & security

Stateful policies block rogue agent actions while preserving task utility

Tiffany et al. present Sapien, a policy engine that generates a task-specific security policy before an AI agent begins work, then checks each tool call against that policy. On two agent-security benchmarks, it ruled out 62 to 95 percent of attacks under a fully hijacked-agent threat model, while keeping task utility within a few percentage points of an unconstrained agent.

yesterday

Agents

Agentic verification improves outputs on long-horizon workspace tasks

Zhang et al. test whether a fixed language model can verify its own long-horizon work without reference answers, grading rubrics or a stronger judge. Across five workspace benchmarks and two frontier models, their harness outperformed the evaluated selection baselines, while evidence-backed revision improved average scores over a single rollout by more than six points.

yesterday

All Harnesses & orchestration papers →

Every paper read and written up by the research desk from the daily arXiv scan · threads are maintained lines of inquiry with running syntheses