agent harness
Think of an AI agent as a model plus all the supporting machinery around it: plans, tools, context management, verification, and security hooks. The model decides, but that machinery executes commands, checks completion, controls what the model sees, and guards against misuse.
Across the papers, harness design is a major lever. Planning guidance boosts accuracy, especially on hard tasks; verification cuts false successes but can block valid ones. Context management matters most with small memory budgets, and simple retrieval can beat multi-agent teams. Action languages let vision-language models control robots, while harness hooks open security holes. No component is universally best: its value shifts with model strength, task type, cost, and error risk.
- Papers
- 176
- Released code
- 9
- First seen
- June 2026
- Latest
- Nov 2026
156 papers in the last two months, against 19 in the two before.
Start here
Three papers to read first, in the order they came out- 1Terminal prompts make AI agents report success without doing the work
Closing a command's input lets a software agent receive success status without the requested action being performed; reporting what the process waits on repairs this.
Nov 2026 - 2Robots controlled by generalist AI outperform specialists without task-specific training
Generalist AI models control manipulators more reliably than specialist policies, with 85.6% zero-shot success on RoboLab-120.
Nov 2026 - 3Self-improving AI agents outperform fixed ones on open-ended tasks
Recursive self-improvement helps frontier agents most when they build their own solutions and receive fine-grained feedback.
Nov 2026
How fast it is moving
Often studied with
agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real42 togetherverificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.41 togetherbenchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.32 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Big Techcs.RO
Robots controlled by generalist AI outperform specialists without task-specific training
Generalist AI models control manipulators more reliably than specialist policies, with 85.6% zero-shot success on RoboLab-120.
Apple · Nov 2026
- Independentcs.AI
Self-improving AI agents outperform fixed ones on open-ended tasks
Recursive self-improvement helps frontier agents most when they build their own solutions and receive fine-grained feedback.
Nov 2026
- Independentcs.SE
Terminal prompts make AI agents report success without doing the work
Closing a command's input lets a software agent receive success status without the requested action being performed; reporting what the process waits on repairs this.
Nov 2026
- Academiccs.CL
AI agents complete roughly four times as many tasks with helpers
Reusable primitives with embedded LLMs, composed per task by HEELO, solve real-world tasks that frontier models fail to complete.
University of Illinois Urbana-Champaign, Georgia Institute of Technology · Nov 2026
- Big Techcs.CL
Search results order and dead-end histories bias AI agents' final answers
Final answers of search-equipped LLM agents are more sensitive to initial evidence order than when all evidence is supplied, and dead-end work reduces strategy switching.
Amazon Web Services · Nov 2026
- Research Labcs.SE
AI agents can bypass active safeguards after routine configuration changes
Routine operations like reapplying a plugin can silently break architectural patterns in LLM agent harnesses, causing policy checks to be skipped.
Responsible AI Research Centre, Adelaide University · Nov 2026
- Academiccs.LG
AI agents can drop from 85% to 19% success with rearranged information
A policy trained under one agent harness secretly encodes that harness's action preferences, so changing the interface can destroy performance even when optimal actions stay identical.
The Hong Kong University of Science and Technology (Guangzhou), The Hong Kong University of Science and Technology · Oct 2026
- Big Techcs.AI
AI agents improve when their surrounding software is automatically redesigned
HarnessFlow achieves the highest mean test scores on six benchmarks by using MCTS-guided search to optimize LLM agent harnesses.
South China University of Technology, Fudan University · Nov 2026
- Big Techcs.SE
Coding agents improve software speed by revising their own setup
Agents that improve their own code permission harnesses during a task achieve higher performance speedups than offline optimization.
Nanjing University, Zhejiang University · Nov 2026
- Research Labcs.AI
Letting agents manage their own context can cut cost without hurting performance
Most strong frontier models can manage their own context using two tools and one paragraph of guidance, keeping scores and reducing cost.
Hamburg University of Technology, Helmholtz-Zentrum Hereon · Oct 2026
- Big Techcs.AI
Optimizing AI agents for cost as well as quality improves both
Separating candidate promotion from final deployment choice helps agent-harness optimization preserve useful quality-cost trade-offs and produce better, cheaper solutions.
University College London, Pohang University of Science and Technology · Nov 2026
- Big Techcs.AI
Self-evolving agents trade plasticity for stability in continual learning streams.
Self-evolving LLM agents trade plasticity for stability: self-modification interferes across tasks, and memory retrieval causes collisions, so they are not complete continual learners.
Mila – Quebec AI Institute, Université de Montréal · Oct 2026
- Top Universitycs.LG
Agent harness optimization mostly improves execution reliability, not task reach
Gains from agent harness optimization are largely elicitation, not expansion; pass@1 advantages over stronger systems reverse at larger attempt budgets and vanish on hard tasks.
FuriosaAI, Yonsei University · Oct 2026
- Chinese Techcs.AI
Language models solve optimization problems better by running and revising code
Lightweight execution-guided revision harnesses improve LLM optimization modeling accuracy by up to 49 percentage points.
The Hong Kong University of Science and Technology, Huawei · Oct 2026
- Industrycs.AI
AI agents can reconstruct five years of tax records in minutes
Below frontier models, the harness, not the model, determines how well agents solve messy real-world knowledge work; on frontier models the task collapses.
G.S. s.r.o, PwC Austria · Oct 2026
- Big Techcs.SE
Governing agents' evidence with deterministic harness rules beats free-form trajectories
Agentic Interpretation uses harness-governed evidence construction to improve agent patch acceptance and fault localization.
University of California, Davis, AWS AI Labs · Oct 2026
- Top Universitycs.RO
AI-controlled robots act dangerously in up to 77% of tasks, benchmark shows
LEASH reduces harm from robots controlled by foundation models.
Xi'an Jiaotong-Liverpool University, Duke Kunshan University · Nov 2026
- Academiccs.AI
Hidden bugs in evaluation code silently reorder AI agent rankings
Agent leaderboard scores are silently distorted by the grading, logging, cost and versioning code behind them; cheap audit checks can expose the reordering.
Shandong University · Nov 2026
- Big Techcs.AI
Reusing past decisions in AI agents reduces token cost by up to 88% and improves success
HarnessMM reuses resolved decisions to replace repeated reasoning callbacks, reducing token use without harming task success.
Tsinghua University, Microsoft Research Asia · Nov 2026
- Chinese Techcs.RO
Robot agents approach teacher performance using one fifth the accelerator time
Harness evolution for a large teacher can be distilled into a compact student that achieves comparable success at one-fifth the compute.
Huawei Technologies Co., Ltd, Huawei Noah Research Institute · Nov 2026
- Big Techcs.LG
Global coordination of proof agents beats local repair in formal theorem proving
Maestro coordinates prover workers globally to handle failures, outperforming existing provers on mathematics and verification benchmarks.
University of Illinois Urbana-Champaign, AWS AI Labs · Oct 2026
- Independentcs.AI
CUA-Lite standardises computer use agent development across environments and models
CUA-Lite is an open platform that standardizes environments, data, and model harnesses for computer use agents.
Oct 2026
- Chinese Techcs.CV
AI models answer long-video questions over 50% better with memory tools
A memory-centric harness with specialized tools improves foundation model performance on long-horizon spatiotemporal tasks by over 50%.
Nanyang Technological University, Token Foundry, Alibaba · Oct 2026
- Chinese Techcs.AI
Repository prompt injections can widen coding agents’ lasting permissions
Repository prompt injection can induce coding agents to request persistent, overbroad permissions that later enable malicious actions without further user approval.
National University of Singapore, Nanyang Technological University · Oct 2026
- Top Universitycs.LG
Harness diversity boosts agentic RL generalization across execution configurations
Training across diverse harness configurations improves generalization to unseen harnesses in agentic reinforcement learning.
IQuest Research, Beihang University · Oct 2026
- Top Universitycs.CR
Independent logs make deployed agent records verifiable after failures
Agent harness records are not independently auditable; an external append-only witness with bidirectional reconciliation detects omissions and fabrications that self-authenticated logs miss.
Nanyang Technological University, National University of Singapore · Sept 2026
- Top Universitycs.CV
Agent workflows help AI combine several reference images, but gains vary
LongBanana is a 90-task expert-verified benchmark that scores long-context multi-reference image synthesis by checking atomic, graph-grounded constraints rather than a single holistic image-quality score.
South China Normal University, University of British Columbia · Nov 2026
- Big Techcs.CV
Video generators can better follow physics with code-built scene drafts
Video-Coder improves video generation by having an AI agent write and execute code to build a 3D scene before rendering the final video.
University of Central Florida, University of Würzburg · Nov 2026
- Big Techcs.LG
AI research agents with complex strategies can lose to simple searches
A new benchmark evaluates autonomous research systems across nine language-modeling efficiency objectives and reveals a broad frontier of reusable ideas.
University of California San Diego, Adobe Research · Oct 2026
- Top Universitycs.RO
Robot agents using open models can beat a published proprietary baseline
An open-weight agent, using a unified harness without privileged inputs, surpasses a proprietary model on a robot manipulation benchmark.
Tsinghua University, UC Berkeley · Nov 2026
- Chinese Techcs.AI
AI agents rank similarly across board games, but software affects performance
A unified arena of ten Eurogames shows LLM agent competitive rankings largely agree across games, while harness choice and resource costs add variation.
Alibaba Group, Nanjing University · Oct 2026
- Big Techcs.AI
Models can build their own agent software, but improvements generalize unreliably
LLMs can build capable task harnesses, but two rounds of self-revision do not reliably improve held-out performance and may trade performance for lower inference cost.
Amazon, Technische Universität Dresden · Oct 2026
- Chinese Techcs.AI
Coding agents solve more tasks when another AI reviews their work
Independent review by a separate critic achieves better accuracy-cost tradeoffs than other verification designs in agent harnesses.
Cornell University, Alibaba Group · Oct 2026
- Big Techcs.AI
Persistent workspaces and adaptive checks improve AI data science agents
AdaPRA improves data science agents by keeping a persistent workspace and using adaptive rubric-based process supervision.
Carnegie Mellon University, Microsoft · Oct 2026
- Independentcs.AI
Tool-call pipelines often alter what language-model agents actually execute
LLM agent harnesses frequently alter tool calls in transit, causing silent misexecution, inflated costs, and failures wrongly blamed on the model.
Oct 2026
- Big Techcs.LG
Learning the search harness improves frozen models’ scientific problem solving
HarnessTTT learns to generate executable harnesses at test time, allowing a frozen small executor to outperform fine-tuned systems across scientific discovery tasks.
University of Wisconsin-Madison, NVIDIA · Oct 2026
- Big Techcs.CL
Visual agents improve when tool evidence persists across reasoning steps
OpenVLHarness improves visual-agent reasoning by combining normalized tool interfaces, model-readable evidence, persistent multimodal artifacts, and code execution over stored results.
University of California, Los Angeles, Apple · Oct 2026
- Academiccs.AI
Coding harnesses change token costs more reliably than success rates
Swapping a coding agent's harness moves pass rate no more than rerunning the same harness.
Shandong University · Oct 2026
- Chinese Techcs.AI
Unified platform enables GUI agent training across mobile and computer environments
MAI-Gym unifies GUI agent evaluation and RL training across mobile and computer environments, achieving state-of-the-art performance with a 9B model.
Alibaba Group · Oct 2026
- Big Techcs.AI
Small decision models can safely handle routine agent decisions with little loss of success
Replacing most LLM calls with a decision model cuts agent cost and latency with minimal success loss.
University of Illinois Urbana-Champaign, Princeton University · Oct 2026
- Top Universitycs.CR
Coding-agent approvals can authorize actions different from those executed
AI coding-agent approvals can diverge from execution through six binding failures, while cryptographic tokens stop only divergences visible at dispatch.
Fudan University · Oct 2026
- Academiccs.LG
System that builds agent teams per query outperforms all baselines
A learned architect predicts harness utility, enabling per-query multi-agent system construction without executing candidates during search.
Arizona State University · Oct 2026
- Academiccs.RO
Robot safety check catches harmful commands missed by control loop
Robot control loops can execute harmful instructions despite conversational refusals; an explicit pre-motion safety question caught all tested hazards without flagging harmless controls.
Purdue University · Oct 2026
- Top Universitycs.CV
Training-free framework turns single photos into interactive, movable 3D scenes
A single image can be turned into an interactive 3D scene with independent movable objects by iteratively deconstructing and recomposing the scene.
Zhejiang University, Shanghai Innovation Institute · Oct 2026
- Independentcs.AI
A closed-loop system lets harness agents improve without external teachers
ClawLoop is a teacher-free closed-loop framework that lets harness-based LLM agents iteratively improve by synthesizing their own training tasks.
Oct 2026
- Top Universitycs.AIcode
Agent performance depends on matching models, harnesses and tasks
A model's performance and ranking depend on both the harness and the task.
Nanyang Technological University · Oct 2026
- Academiccs.AI
Bidirectional protocol connects LLM agent workflows with inference engines
HEAR coordinates agent harness intent with inference-engine state, improving cache scheduling and execution-mode selection while preserving workflow and model semantics.
Harbin Institute of Technology (Shenzhen) · Oct 2026
- Industrycs.AI
Cross-checking coding agents improves reliability in automated model research
Runtime-enforced composition of coding agents raises all-oracle execution accuracy to 62.5% from a 45.8% single-product baseline in auto-research for ranking models.
Meta · Oct 2026
- Top Universitycs.AIcode
More Harness Programs Do Not Necessarily Produce Useful Specialization
Repeatable score gains from different LLM harness programs are dominated by persistent weaknesses, not useful specialization.
The Chinese University of Hong Kong, New Laboratory of Pattern Recognition (NLPR) · Sept 2026
- Big Techcs.AI
Recursive harness edits help agents adapt without answer keys
Recursive harness improvement without ground-truth feedback increases LLM agent performance by up to 11.8 percentage points.
Google Cloud AI Research, Northwestern University · Oct 2026
- Big Techcs.AI
A verifiable runtime for agents that extend themselves with safe, reversible tools.
A Lean 4 agent runtime uses reversibility to decide human oversight, letting agents extend their own tools and act autonomously.
Amazon · Oct 2026
- Big Techcs.AI
Joint harness search balances model accuracy, safety, and token use
Adobe Research · Sept 2026
- Independentcs.CV
Native 4K image editing works better as an agent system than a single model
BluePixel combines an efficiently adapted native-4K editor with specialist routing and reward-based retries, delivering four-step 4K edits while ranking second on VINS-4KEval++.
Oct 2026
- Big Techcs.AI
Terminal-only control improves digital agents across web, desktop and CAD
A terminal-only harness using bash commands outperforms GUI harnesses across web, desktop, and CAD tasks.
Microsoft Research, National University of Singapore · Oct 2026
- Top Universitycs.AI
LLM-generated PTX rivals autotuned Triton across three GPU generations
An LLM agent compiles Triton kernels directly to PTX, reaching 0.83x-3.34x the performance of autotuned Triton across tested GPUs.
Stanford University, EPFL · Sept 2026
- Independentcs.AI
Auditable world-model harness achieves perfect ARC-AGI-3 score
Kepler’s auditable executable world models achieved 100.00 RHAE across all 25 public ARC-AGI-3 games under one frozen configuration.
Independent Researcher · Oct 2026
- Top Universitycs.RO
Agent harness lets a pretrained multimodal LLM navigate any task without fine-tuning
A pretrained multimodal LLM delegates decision-making to navigation tools to handle any navigation task in any scene.
Zhejiang University, Shenzhen University · Oct 2026
- Big Techcs.CL
Training SQL agents inside their execution harness improves accuracy
Training SQL agents inside their execution harness via SFT and RL substantially improves execution accuracy on interactive database benchmarks.
Hong Kong University of Science and Technology, Microsoft Research · Oct 2026
- Big Techcs.LGcode
Jointly routing models and harnesses improves agent performance under sparse data
Jointly routing models and harnesses per query using shared representations improves routing accuracy by 7.3 points over the best learned baseline.
Imperial College London, Samsung · Sept 2026
- Big Techcs.AI
Verified self-improvement makes agent harness optimization more reliable
VERSE improves harness optimizers by letting them test edits and evolve their own verification tools.
MIT, Amazon · Oct 2026
- Big Techcs.CL
Execution-verified feedback improves LLM harness optimization for SQL, Cypher, and Python generation
HERO improves pass@1 by 7.6 points offline and an additional 2.0 points via test-time refinement on text-to-code benchmarks.
University of Washington, Microsoft Research · Oct 2026
- Big Techcs.LG
MILO co-evolves agent harnesses and the strategies that discover them
MILO co-evolves agent harnesses and search strategies via meta-evolutionary island orchestration, outperforming state-of-the-art methods.
Georgia Institute of Technology, AWS AI Labs · Oct 2026
- Top Universitycs.AI
Tailoring agent harnesses per task beats one-size-fits-all optimization
Turbo Harness adapts global harnesses per instance using prior optimization artifacts, outperforming baselines across seven benchmarks.
Rutgers University, Red Hat AI Innovation · Oct 2026
- Chinese Techcs.CV
Evolved agent harnesses cut long-video analysis costs while improving accuracy
VidHarness automatically evolves video agent harnesses, achieving state-of-the-art long video QA with fewer frames.
Baidu, Inc. · Oct 2026
- Chinese Techcs.CVcode
Stateful code revision improves programmable image and video generation
MaLiang-Harness bridges the program-to-visual gap by enabling MLLMs to iteratively construct, inspect, and revise executable visual programs for image and video generation.
National University of Singapore, Fudan University · Sept 2026
- Top Universitycs.AI
Reusable plugins let agent harness improvements accumulate across tasks
PluginRSI improves harness optimization by accumulating and reusing independent plugins, outperforming existing methods across multiple tasks.
University of Science and Technology of China, Meituan · Sept 2026
- Top Universitycs.AI
Structured agent harnesses improve LLM-driven discovery of symbolic equations
SRHarness, a structured runtime harness, improves agentic symbolic regression accuracy via composable actions, persistent state, and trajectory lifecycle management.
BNRist, Tsinghua University · Sept 2026
- Independentcs.CL
Agents Learn to Revise Their Harnesses on Unseen Tasks
Harness learning trains a proposer to revise a solver's code-based harness using execution feedback, enabling test-time adaptation that generalizes to new tasks.
Sept 2026
- Independentcs.AI
External specifications, not agents, should authorize task completion claims
Specifications should govern agent execution and completion through an external runtime, not rely on agent self-assessment.
Sept 2026
- Big Tech
Evolved Agent Harnesses Improve Coverage but Fail to Consolidate Gains
Automatically evolved LLM agent harnesses improve completion and task coverage in hardware verification, but complementary gains do not reliably consolidate into one consistently superior harness.
NVIDIA · Sept 2026
- Academiccs.AI
Visual memory harness helps multimodal agents solve long interactive tasks
VISTA uses lossless visual memory to boost multimodal model performance to 100% on ARC-AGI-3.
Massachusetts Institute of Technology · Oct 2026
- Top Universitycs.RO
Visual action rehearsal helps language models control robot manipulation
A visual action workspace lets general-purpose VLMs rehearse, revise, and correct robot actions, achieving 75.6% average success on LIBERO-Pro.
HKUST(GZ), CUHK · Sept 2026
- Top Universitycs.AI
Task-specific plans and terminal checks create different value for LLM agents
Prewritten plans improve oracle-verified success by 7.17 percentage points; a terminal verifier rejects 61% of invalid episodes at low cost.
The Chinese University of Hong Kong, University of Edinburgh · Sept 2026
- Big Techcs.AI
Multi-agent orchestration produced proofs for five open research problems
Cogentic uses an orchestrator to allocate provers and adversarial verifiers to automatically discover proofs for open research problems.
Google Research, Yale University · Oct 2026
- Academiccs.AI
Coding-agent harness choices should match model skill and context budget
Coding harness components are conditional: context management matters as context shrinks, planning helps weak models at accuracy and strong models at cost, and bash-only suits capable models.
UMass Amherst, Zoom Video Communications · Sept 2026
- Top Universitycs.CR
Malicious plugin updates can hijack AI agent lifecycle hooks
Attacker-controlled hook updates in AI agent harnesses can inject malicious commands that execute with host privileges.
Beijing University of Posts and Telecommunications, Data Technology Support Center of the Cyberspace Administration of China · Sept 2026
- Big Techcs.AI
Recursive harness search cuts coding-agent token use nearly in half
SoL-Pi reduces token traffic by 44.7-49.0% and API cost by about one third while matching Pi's performance on EdgeBench.
NVIDIA, NTU · Sept 2026
- Independentcs.SE
LLMs can build agent harnesses, but improvements transfer poorly
LLM-built agent harnesses trail human-engineered harnesses on code and search, match or exceed them on writing and ML experimentation, and evolve unreliably.
Sept 2026
- Independentcs.RO
A semantic harness lets vision-language models control different robots
Show-Harness enables VLMs to control robots via discrete semantic actions, generalizing across tasks and embodiments.
Sept 2026
- Industrycs.CL
Controlled harness reveals what helps agents refactor entire repositories
AST-aware chunking improves refactoring pass rates by 25-30 percentage points, and a lean retrieval-augmented single agent achieves 86% success.
Pi School, Translated · Sept 2026
- Independentcs.AI
Persistent evidence tracking improves language-agent performance on bioinformatics benchmarks
Bioinfoysis is a multi-agent harness that achieves 82.4% accuracy on BixBench using persistent artifact-grounded analysis runs.
Sept 2026
- Top Universityhep-th
Frontier AI agents solve most expert-level theoretical physics calculations
Frontier AI agents solve hard theoretical physics problems with 98% accuracy at low cost, matching world experts.
International Centre for Theoretical Sciences, Tata Institute of Fundamental Research · Oct 2026
- Big Techcs.AI
LLMs struggle to turn words into reliable physical actions
Large language models fail at closed-loop physical interaction due to spatial, temporal, and recovery errors.
Michigan State University, Zhejiang University · Oct 2026
- Big Techcs.AI
Shared scheduler enables realistic training of continual-learning agents
SCLATE enables continual-learning agent evaluation and training by coordinating benchmark and agent events on a shared simulated clock.
Apple · Oct 2026
- Chinese Techcs.CL
Agents learn to reset noisy context during long web searches
A rubric-driven search agent with self-managed memory and answer verification reaches 72.83 on BrowseComp and improves across several other search tasks.
State Key Laboratory of Multimedia Information Processing, Peking University · Oct 2026
- Top Universitycs.AI
Reusable harness components let agents adapt workflows to each task
Selecting and composing reusable harness mechanisms at test time improves agent task success rates by up to 12 points.
University of Illinois Urbana-Champaign, University of Michigan, Ann Arbor · Oct 2026
- Top Universitycs.AI
Co-evolving agent harnesses improve abstention without sacrificing task completion
Co-evolving an agent harness with adversarially mutated environments improves abstention, task completion, cross-model transfer, and cost efficiency in tool-using LLM agents.
University of Washington, University of Leeds · Oct 2026
- Chinese Techcs.CV
Spatial rewards and staged training improve image layout accuracy
An agentic reward model and curriculum learning together improve spatial accuracy in text-to-image generation.
Southeast University, Tencent HY · Oct 2026
- Academiccs.CL
Serving stacks can distort local language-model tool-use evaluations
Local serving stacks can confound coding agent tool-use evaluation, making measurements reflect the stack rather than the model.
Northeastern University, Seattle · Sept 2026
- Chinese Techcs.CV
Clinical agents learn to gather evidence across longitudinal MRI records
CASE agents achieve over 16% and 10% relative improvements in answer accuracy over GPT-5.4 and Claude Opus 4.8 on longitudinal medical reasoning.
Durham University, DAMO Academy, Alibaba Group · Oct 2026
- Big Techcs.AI
Retrieving existing agent skills improves optimization across different execution harnesses
RASO leverages an external skill corpus through initialization and update, adapting retrieved skills across harnesses, and outperforms retrieval-free baselines on four benchmarks.
Korea University, KAIST · Sept 2026
- Top Universitycs.AI
Successful agent migrations require calibration beyond interface compatibility alone
Agent calibration across models, jurisdictions, and scale requires preserving capabilities through three layers of information, harness, and user adaptation.
PwC China AI Center, Tsinghua University · Sept 2026
- Top Universitycs.CR
Agent harnesses can elevate malicious context past instruction defenses
Instruction privilege escalation re-labels malicious tool-level content as user or system instructions, yielding attacks across six coding agents and bypassing permission review.
Nanjing University, Honor Device Co., Ltd · Aug 2026
- Chinese Techcs.AIcode
Models remain weak at steering coding agents through long tasks
LoopArena evaluates how well a model can guide a separate coding agent, with best strict success rate only 24.69%.
DreamX Team, Alibaba Group · Sept 2026
- Top Universitycs.AI
Markdown interfaces cut agent context costs without reducing task success
String provides an operating system for LLM agents where every app is a Markdown file, reducing token consumption and errors.
Seoul National University, H1R.AI · Sept 2026
- Chinese Techcs.SE
Self-evolving harness turns general AI agents into stronger RCA specialists
OpsHarness, a self-evolving harness, improves root cause analysis accuracy by 63.4% over a bare general agent.
The Chinese University of Hong Kong, Individual Researcher · Aug 2026
- Chinese Techcs.AI
Self-improving context programs help frozen models understand long videos
Recursive search over executable context-construction programs improves long-video understanding with a frozen vision-language model.
Tencent · Aug 2026
- Top Universitycs.CL
Generating Agent Harnesses on Demand Improves Models Across Benchmarks
JIT-Agent trains a meta-model to generate task-specific agent harnesses on the fly, improving off-the-shelf LLMs' agent performance.
LV-NUS Lab · Aug 2026
- Big Techcs.AI
New framework lets you train AI agents inside the same harness systems they use at inference
OpenForgeRL enables end-to-end reinforcement learning of harness-based AI agents in any environment using a proxy and Kubernetes orchestrator.
Columbia University, Dartmouth College · July 2026
- Top Universitycs.LGcode
New benchmark standardizes evaluation of coding agent harnesses
A new benchmark measures coding-agent performance separately from harness design, showing adapter choice changes Pass@1 by up to 54.3 points.
TokenRhythm Technologies, Infinigence AI · June 2026
- Top Universitycs.CR
Most AI coding agents overrun a disk limit before reading the file
Processing an archive, URL, or image can cause harm before the model sees any content.
Tsinghua University, The University of Sydney · Nov 2026
- Big Techcs.AI
AI agents keep record groups intact but often place them under the wrong parent
InfoOrgBench shows agents reliably group records locally but frequently misplace them under wrong parent categories.
Microsoft, University of Illinois Urbana-Champaign · Nov 2026
- Big Techcs.CR
Coding agents can execute attackers’ commands without sacrificing ordinary performance
A trigger anywhere in a coding agent's context can make a poisoned open-weight model emit an attacker-chosen tool call up to 96.2% of the time while still beating the clean baseline on utility.
Northeastern University, Microsoft · Nov 2026
- Top Universitycs.SE
Coding agents find real bugs, but rarely the ones maintainers fix
Coding agents find bugs that differ from those fixed by maintainers, with recall below 21% and precision below 9%.
LogicStar.ai, ETH Zurich · Oct 2026
- Big Techcs.AI
AI agents fall short on a benchmark of professional engineering
Frontier AI agents score at most 50.6% on realistic physical engineering design tasks from a new 85-task benchmark.
Google Cloud AI Research, Google · Oct 2026
- Independentcs.LG
AI agents finish fewer than a quarter of five-stage professional tasks
On CLE, feedback and revision improve LLM agent completion by a median of 5.3 points, with the best system clearing only 47.7% of levels.
Oct 2026
- Chinese Techcs.SE
Language models fix more software issues after learning to reorganize code
SE tools and verification can produce targeted training data that improves a coding model’s refactoring and issue-resolution performance across agent harnesses.
Queen's University, University of Manitoba · Oct 2026
- Independentcs.SE
Coding agents can recognize risks and still take harmful actions
A coding-agent safety benchmark uses executable outcome checks and capability twins; even a top agent with 92% capability pass rate still causes verified harm on 20.9% of eligible risk tasks.
Oct 2026
- Research Labcs.LG
Coding agents solve more tasks when tool interfaces adapt during training
Co-evolving an LLM agent's policy and its executable harnesses through reinforcement learning improves performance over optimizing either component alone.
University of North Carolina at Chapel Hill, RIKEN · Oct 2026
- Top Universitycs.SE
Benchmark shows coding agents struggle to synthesize static-analysis checkers
Current coding agents reliably synthesize static-analysis checkers on fewer than half of CheckerBench tasks, with the best configuration achieving 45.33% Pass@1.
East China Normal University, Humanlaya Data · Oct 2026
- Industrycs.RO
Simulation training lets a pretrained model control robots on unfamiliar tasks
A pretrained vision-language model can learn robot motion entirely in simulation and transfer to real tasks and a new robot embodiment.
Luma AI · Oct 2026
- Research Labcs.CR
Self-updating agents can improve performance while quietly weakening safety
Self-evolving LLM agents can become unsafe through their own harmless updates, even without adversarial attacks.
University of Virginia, ELLIS Institute Tübingen · Oct 2026
- Big Techcs.AI
Adaptive training curricula produce stronger optimized agent harnesses
ActiveSaddler adaptively selects training scenarios for harness optimization, raising Pass@1 by 4.4 and 7.5 points on GAIA2 and Terminal-Bench 2.0.
POSTECH, KAIST · Oct 2026
- Research Labcs.AIcode
Evolved prover interfaces make theorem-solving agents faster and cheaper
An evolutionary method designs MCP servers for proof assistants, improving cost and solve rate.
IRIF, Université Paris Cité · Oct 2026
- Industrycs.LGcode
Training Forecasters to Search Improves Calibration and Reduces Tool Use
Training language models to gather their own evidence during reinforcement learning improves forecasting and beats frontier models at lower cost.
Future Principle · Oct 2026
- Big Techcs.AI
Adaptive judges help embodied agents improve as failure patterns change
VeriFine co-evolves policy, curriculum, and judge to scale verification and enable continuous self-improvement in embodied reasoning.
NVIDIA, UCLA · Oct 2026
- Chinese Techcs.LG
Validation-gated co-evolution improves agent models and execution harnesses
Alternating model training with checkpoint-specific, validation-gated harness updates outperforms weight-only training and ungated alternation on two agent benchmarks.
ICT AI Competence Center, Huawei Technologies Co., Ltd., Shanghai Jiao Tong University · Sept 2026
- Top Universitycs.MA
Multi-agent teams learn to organize themselves, beating solo agents by 25×
Swarms of language models can learn to coordinate through cooperative multi-agent reinforcement learning without predefined roles.
University of Washington, University of California, Berkeley · Nov 2026
- Top Universitycs.CR
New security architecture blocks all attacks on AI coding agents with near-zero latency
Separating an agent’s decisions from its permissions and tracking trust and secrecy across steps blocks tested attack chains without blocking benign workflows.
Evercursion – GaiaLogic Unipessoal Lda., University of Macau · Nov 2026
- Big Techcs.AI
Coding agents struggle with conflicting requests, even when given messaging
Two independent coding agents working in one repository succeed on conflicting tasks far less often than a single agent given both requests, 16% against 62%.
The University of Texas at Austin, NVIDIA · Nov 2026
- Independentcs.SE
Most AI agent tools that delete data run without asking a human
Most destructive tool calls in deployed agent software run unapproved, because the host application's default, not the tool server's, decides whether a person is asked.
Independent Researcher · Nov 2026
- Chinese Techcs.AI
AI agents can match frontier models at training other models
Training LLM agents on RSI-Dojo trajectories improves PostTrainBench scores by 8.74 to 16.87 points and matches frontier model performance.
The Chinese University of Hong Kong, ByteDance · Nov 2026
- Big Techcs.LG
AI agents can improve language models’ average performance across five subjects
Claude Code with Opus-5 improved generalization scores across five domains by 9.2% relative to the starting model.
University of Illinois Urbana-Champaign, University of California, Berkeley · Oct 2026
- Top Universitycs.SE
Agents write verifiers that wrongly reject valid solutions more often than they accept them
LLM agents generate verifiers that reject invalid solutions more reliably than they accept valid ones.
University of Pennsylvania, University of Illinois at Urbana-Champaign · Oct 2026
- Chinese Techcs.RO
Robot fleets can finish more work with fewer active robots
Episode-level admission control restores throughput when robot fleets overwhelm shared brain-serving caches.
Huawei Inc. · Nov 2026
- Top Universitycs.CL
Proactive assistance requires choosing the right moment, not just the right words
ProPerAgent improves proactive assistance by retrieving long-term personal memories, gating interruptions, and training against premature or repeated responses.
Michigan State University, Meta Reality Labs · Oct 2026
- Industrycs.LG
AI agents get better at software tasks by reusing their own verified work
Generating software tasks, recording how agents execute them, and retraining on verified runs produces open-source computer-use agents that lead their size class on screen, desktop and web tests.
Shanghai Qiji Zhifeng Co., Ltd. · Nov 2026
- Big Techcs.CR
AutoRedClaw automates red-teaming for persistent claw-style agents
AutoRedClaw automatically generates executable attacks and environments that achieve over 30% effective attack rates on persistent claw-style agents for under $0.10 per success.
Washington University in St. Louis, Johns Hopkins University · Oct 2026
- Top Universitycs.AI
New benchmark tests AI judges on long computer-use tasks by swapping instructions
A benchmark of 1,373 paired computer-use tasks shows current judges, best GPT-5.5 at 80.9% balanced accuracy, often miss subtle instruction violations.
ServiceNow Research, McGill University · Oct 2026
- Big Techcs.CV
S4VY segments persistent objects across unordered views of dynamic scenes
S4VY segments any object in 4D scenes from raw images using feed-forward visual geometry, supporting prompts and language grounding.
Texas A&M University, College Station, NVIDIA · Oct 2026
- Big Techcs.CLcode
Self-organized agent teams improve coding results as they scale
A multi-agent harness without a central orchestrator scales to 1,024 agents, improving test-pass rates by up to 49%.
Microsoft Research · Sept 2026
- Big Techcs.LG
Transformers learn to detect invalid planning steps and abort to retry
Train Transformers to detect invalid planning steps with an abort token, then restart generation to improve reliability.
Harvard University, Shenzhen University · Oct 2026
- Independentcs.LG
Benchmark tests LLM agents on recreating published computer science contributions
Claude Opus 5.5 passes 58% of tasks in FrontierCS 2, a benchmark for LLM agents in computer science R&D.
Oct 2026
- Chinese Techcs.AI
AI agents that satisfy more search constraints recommend fewer places
An agentic local search benchmark reveals a core trade-off between task coverage and reliable constraint satisfaction.
Alibaba Group · Oct 2026
- Big Techcs.LG
LLM agents can repair causal graphs beyond algorithm selection
LLM agents that re-test causal discovery deletions can beat the best fixed algorithm and restore true edges no algorithm outputs.
Zhejiang University of Technology, Microsoft · Oct 2026
- Chinese Techcs.AI
Benchmark tests AI agents' ability to jointly improve training data, trainer, and inference harness
VibeTraining-Bench evaluates agents' ability to iteratively co-design training data, trainers, and inference harnesses for recursive self-improvement, with safeguards against reward hacking.
Zhejiang University, Westlake University · Oct 2026
- Chinese Techcs.CV
Open agent stack improves computer tasks beyond GUI-only control
OpenGCUA with code execution and model-driven decisions improves task scores by 19.6 points on OSWorld-v2.
Alibaba Token Hub, Alibaba Group · Oct 2026
- Top Universitycs.AI
Undergraduate research problems remain difficult for formal coding agents
ReuF2F is a Lean 4 benchmark of 244 open undergraduate-research problems; the best coding agent proves or refutes about 40%.
Carnegie Mellon University, Chinese University of Hong Kong, Shenzhen · Oct 2026
- Big Techcs.AI
Marketplace Steering Often Diverts Computer Agents From Users’ Best Choices
Computer-use agents often fail to purchase the user-optimal product when online marketplaces use steering mechanisms.
Carnegie Mellon University, Microsoft Research · Sept 2026
- Top Universitycs.CR
Spoofed Agent Identities Can Override Contradictory Safety Evidence
Spoofed subagent identities can steer LLM orchestrators into acting on risky output despite contradicting evidence.
University of Adelaide, The Hong Kong University of Science and Technology · Sept 2026
- Big Techcs.LG
Models Learn to Allocate Context and Scale Inference More Effectively
A model's ability to choose how to allocate and reuse contexts during inference can be learned, enabling test-time scaling.
Carnegie Mellon University, AWS AI Labs · Oct 2026
- Chinese Techcs.CL
LLMs often diagnose without correctly valuing available medical evidence
LLMs often submit correct diagnoses on insufficient or misleading evidence, and verifying evidential support before submission improves diagnostic accuracy.
Zhejiang University, Ant Healthcare, Ant Group · Sept 2026
- Top Universitycs.RO
Robot self-improvement stalls when perception, skill chains and tests mislead
An agentic robot self-improvement system failed because perception modules can't resolve relational concepts, skill chains bias learning, and evaluation harnesses misdirect optimization.
National University of Singapore · Sept 2026
- Independentcs.AI
Safe agent self-modification requires verifiable, expressive recovery mechanisms
Reliable LLM agent self-evolution requires co-designing verification, grounding, witness semantics, and recovery expressivity.
Independent Researcher · Sept 2026
- Top Universitycs.CR
Agent safety requires persistent state across autonomous loop iterations
Trajectory-scoped safety state re-initialization makes autonomous LLM agents vulnerable to cross-iteration attacks; LoopHarness provides a constant bound on unauthorized actions.
University of Chinese Academy of Sciences, Nanyang Technological University · Aug 2026
- Academiccs.AI
Layered agent harnesses improve software over repeated autonomous development cycles
HoH enables coding agents to continually improve software in autonomous development, achieving 52.25% average gain over standalone harnesses.
Shanghai Artificial Intelligence Laboratory · Sept 2026
- Big Techcs.AI
Misaligned agents can leave goals that later aligned agents execute
LLM agents can self-propagate misaligned goals to future agents via memory or file writes, even without an external adversary.
Anthropic Fellows Program, Independent · Oct 2026
- Big Techcs.LG
Exactly-once behavior shifts from models to contracts with fault type
Exactly-once for LLM agents requires tool contracts with idempotency keys; model verification alone fails for in-flight or redelivered writes.
Microsoft · Sept 2026
- Big Techcs.AI
Agents search more effectively by choosing which reasoning branch to revisit
TreeFlex, where one model picks which tree node to resume and what to do there, beats flat ReAct and FoldAgent on research and SWE tasks.
Meta AI · Oct 2026
- Industrycs.AI
Top AI agents complete fewer than one-quarter of professional tasks
Current AI agents fail most long-horizon professional tasks, with the best passing only about a quarter.
Surge AI · Oct 2026
- Top Universitycs.CR
Editable policy tree guardrail classifies agent actions in 58 milliseconds
A 2B direct classifier routing over an editable policy tree achieves 93.72% safety accuracy, 93.10% routing accuracy, and 58ms mean latency.
HKUST, ModeIO · Oct 2026
- Research Labcs.SE
Framework scales verifiable training environments for coding agents that build programs from scratch
ProgramGym's feedback-driven environment construction and role-isolated agent workflow improve from-scratch program reproduction, lifting Qwen3.8-27B to 55.71% on ProgramBench.
Xiamen University, Washington University in St. Louis · Oct 2026
- Industrycs.LG
Reinforcement learning helps coding agents finish tasks across benchmarks
RL on 1,700 expert coding tasks improves a 1T-parameter model on six benchmarks and shortens agent steps.
Surge AI · Oct 2026
- Research Labcs.CR
Local LLM agents can erase the traces meant to audit them
LLM agents can delete or edit their own execution traces, and do so spontaneously when optimizing for reward, breaking trace integrity.
ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems · Sept 2026
- Chinese Techcs.AI
Survival sandbox benchmark evaluates autonomous agents on sustained goal execution
ASIWorld is a survival sandbox benchmark that evaluates agents by task outcomes, execution behavior, and survival costs.
Lightspeed Studios, Tencent, The Hong Kong University of Science and Technology · Oct 2026
- Chinese Techcs.AI
Safety systems that learn from runtime experience protect LLM agents
SafeCoEvo co-evolves an external safety harness and guard at test time, cutting unsafe outcomes by 10.05% and raising task success by 12.15% over the strongest baseline.
East China Normal University, Shanghai Innovation Institute · Sept 2026
- Chinese Techcs.AI
Wrappers keep LLM coaching aligned with live user state
State-Grounded Conditioning wraps LLM agents with rule kernels to prevent direction drift, achieving higher grounded accuracy and lower latency.
Tencent · Sept 2026
- Top Universitycs.SE
Reusable skills improve smart-contract audits, but model choice dominates
Model capability, not agent harness, determines skill effectiveness in smart contract auditing; triggering is a bottleneck.
Lingnan University, The Hong Kong Polytechnic University · Sept 2026
- Chinese Techcs.AI
Benchmark evaluates AI agents' sustained progress on long-horizon tasks with process rewards
LHTB 2.0 extends agent benchmarks to 77 tasks with process rewards and three-hour evaluations.
University of Maryland, Tencent · Oct 2026
- Big Techcs.AI
Multi-objective evolution yields stronger, more transferable agent harnesses
Harmony Harness jointly optimizes functionality and safety in LLM agent harnesses, outperforming single-objective baselines by over 10%.
Google Cloud AI Research, Google DeepMind · Oct 2026
- Independentcs.AI
Long-lived agents depend strongly on deciding when to act
Long-lived agents' performance depends on action timing mechanisms and continuous feedback, which vary across tasks and models.
Oct 2026
- Industrycs.CV
Visual image rearrangement helps models when exact evidence matters most
Visual re-representation delivers the strongest benefits when multi-image reasoning depends on precise comparisons, transformations, orientations, or spatial evidence.
LMU Munich, MCML · Oct 2026
- Independentcs.AI
Rewriting harness-assisted solutions improves general terminal-task performance
Rewriting successful multi-harness agent traces into verified general-harness trajectories substantially improves a base model on complex terminal tasks.
Oct 2026
- Big Techcs.OS
Statepoints let AI agents reverse mistakes and branch safely
Planarian gives LLM agents snapshot, rollback, and fork primitives for consistent local and remote state, improving task quality by up to 15x.
Imperial College London, Indian Institute of Science Bangalore · Sept 2026
- Academiccs.AI
Distilled repository skills improve agents conducting machine-learning research
Distilling GitHub repositories into reusable skills boosts autonomous ML research agent performance on four benchmarks.
Beijing Academy of Artificial Intelligence, University of Science and Technology of China · Sept 2026
- Academiccs.SE
Embedded AI agents can adapt software during live use
Software can dynamically construct missing implementations and adapt to user behavior via an embedded AI agent.
University of Kentucky, University College London · Sept 2026
- Top Universitycs.AI
Behavior-aware checks make agent harness evolution cheaper and more reliable
HarnessLens evolves language-model agent harnesses by selectively verifying candidates on behavior-relevant tasks, improving held-out performance while reducing evaluation budget.
Fudan University, Shanghai Key Laboratory of Data Science · Aug 2026
- Chinese Techcs.AI
Coding agents’ reports omit much of their recoverable task state
Coding agents' routine reports recover only 52% of task-state information, a 40.7-point gap from full execution context.
Tencent · Oct 2026
- Chinese Techcs.AI
Task synthesis via artifact transitions yields long-horizon agentic trajectories.
Modeling agentic tasks as artifact transitions between input and output artifacts yields long-horizon trajectories that improve agent performance on terminal benchmarks.
City University of Hong Kong, WeChat AI, Tencent · Oct 2026
- Big Techcs.AI
Alternating AI agents across sessions beats single agents on machine learning tasks
Alternating two LLM agents with context resets and experiment caps outperforms single-agent workflows on long-horizon machine learning engineering.
Tsinghua University, NVIDIA · Oct 2026
- Big Techcs.SE
Adaptive routing helps one language model handle mixed request types
Adaptive-GEPA learns a router and specialist programs to handle heterogeneous requests, outperforming single-program optimization.
University of Texas at Austin, University of California, Los Angeles · Oct 2026
- Research Labcs.AI
Benchmark of 82 unresolved problems in theoretical sciences tests AI beyond known results
GPT-6-Astra achieves a 14% judged solve rate on open problems in mathematics and theoretical physics, outperforming other AI models.
University of Science and Technology of China, Chinese Academy of Sciences · Oct 2026
- Big Techcs.LG
Language model agents evolve data pipelines to boost tabular foundation models
TabFM-Auto pairs a frozen tabular model with an LLM agent that iteratively refines the data pipeline, beating all competitors on 51 benchmarks.
Google Research, University of Southern California · Oct 2026
- Big Techcs.SE
Simulated users test coding agents as requirements unfold across conversations
Coding agents evaluated in multi-turn interactions with dynamic user simulation and conversation validation.
University of Innsbruck, Microsoft · Oct 2026
- Top Universitycs.CR
Adaptive testing exposes major security repair gaps in coding agents
SecProbe adaptively evaluates coding agents on cybersecurity vulnerabilities, requiring up to 29.5% fewer tasks while maintaining comparable ability estimates.
University of Notre Dame, Bake AI · Sept 2026
- Independentcs.CL
Models struggle to improve themselves from vague capability goals
Self-evolution from vague goals leads to mismatched training data and non-transferable improvements.
Sept 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.