Agent Harness Optimization
Techniques for optimizing the code, instructions, and training frameworks surrounding LLM agents to improve performance and reduce cost without changing model weights.
61 papers · 4 months
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
1 reported result from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
May 2026 · Columbia University, Google
Proposing a workflow store to harden AI agents against failure
Proposes a workflow store as an alternative to on-the-fly planning to harden agents against failure.
June 2026 · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Tongyi Lab , Alibaba Group
Evolutionary training harness co-evolves with LLM policies for RL
Proposes co-evolution of training harness and LLM policies, treating the harness as an evolvable component.
June 2026 · City University of Hong Kong, Microsoft Research Asia
Self-supervised method improves agent harnesses using past trajectories
Introduces a self-supervised method to optimize agent harnesses from past trajectories.
June 2026 · Shanghai Innovation Institution, Huawei Technologies Ltd.
Conversational image editing agent learns interpretable tool use
Shows how an agent can be trained to use parameterized tools interpretably via a multitask pipeline.
June 2026 · Stanford University
Decentralized agents with shared context outperform centralized orchestration
Decentralized multi-agent framework with shared context and task queue outperforms central orchestration.
June 2026 · TokenRhythm Technologies, Infinigence AI
New benchmark standardizes evaluation of coding agent harnesses
Benchmark and adapter protocol for evaluating coding agent harnesses, revealing adapter design impact.
June 2026
Latent communication across different LLMs matches or beats text-based transfer at lower compute.
Latent communication via KV-cache alignment reduces compute compared to text-based transfer between heterogeneous LLMs.
June 2026 · Sakana AI
Orchestrator models dynamically combine specialized LLMs into collective intelligence
Orchestrator models generate dynamic, query-adaptive agentic scaffolds to combine specialized LLMs.
June 2026 · NVIDIA Research
Agent framework auto-tunes video diffusion for 2x speedup
Training-free framework using parallel agent modules to auto-tune video diffusion (cache, sparse attention, quantization, etc.).
July 2026 · University of Pennsylvania, Nvidia
Skill evolution improves agent performance in image generation workflows
Skill evolution via typed graph editing in an image generation harness.
July 2026 · Allen Institute for AI, University of Washington
Re-evaluation shows harness evolution for agents may not outperform simple test-time scaling
Re-evaluation showing harness evolution may not outperform simple test-time scaling.
1 further paper
July 2026 · NVIDIA
Python objects become AI agents in new NVIDIA framework
Introduces a model-agnostic Python framework where any Python object can act as an AI agent, simplifying harness construction.
July 2026 · Columbia University, Dartmouth College
New framework lets you train AI agents inside the same harness systems they use at inference
Framework for training agents inside the same harness used at inference.
Aug 2026 · Google DeepMind
Cost-aware model routing matches exhaustive estimation with fewer expensive checks
Models AI routing as search with inspection costs, achieving cost-aware routing across multiple LLM setups.
Aug 2026 · The University of Tokyo
Adaptive task selection cuts the cost of optimizing LLM agent harnesses
Adaptive task selection for optimizing agent harnesses.
1 further paper
Aug 2026 · ServiceNow, Mila
Evolved agent harnesses lift enterprise performance without retraining models
Presents an evolutionary search framework for improving agent harnesses, enhancing enterprise performance without model retraining.
Aug 2026 · AWS
Switching Models Mid-Task Imposes a Cost and Quality Penalty
Quantifies the 'handoff tax' in mid-task model switching, informing harness design decisions about model selection.
Aug 2026 · National University of Singapore, TCG CREST
Bounded citation-graph search improves recall while cutting scholarly search costs
Introduces a bounded citation-graph search method that reduces agent search costs while improving recall.
Aug 2026 · KAIST
Coding agents extend LLM serving simulators with greater fidelity and speed
Simthesizer uses a coding agent to modify a shared simulator, improving harness efficiency for LLM serving workflows.
Aug 2026
Good tool plans can still fail under resource constraints
Introduces PeakBench, a benchmark for evaluating tool plans under resource constraints, adding a new dimension to harness optimization.
released code
30 further papers
Sept 2026 · UMass Amherst, Zoom Video Communications
Coding-agent harness choices should match model skill and context budget
Shows that harness choices (planning, action space, context management) must match model skill and context budget, guiding harness optimization.
1 further paper
Sept 2026 · The Chinese University of Hong Kong, University of Edinburgh
Task-specific plans and terminal checks create different value for LLM agents
Comparing task-specific plans against shuffled text reveals how meaningful planning signals add value, informing instruction design in agent harnesses.
Sept 2026
Voice-agent models diverge most beyond choosing the correct tool
The benchmark isolates decision-making from speech components, allowing targeted analysis of voice-agent reasoning, which is relevant for optimizing harness components.
Sept 2026 · University of Bristol, King Abdullah University of Science and Technology
Formal logic solvers can audit and improve language-model reasoning chains
Proposes a neuro-symbolic framework to audit and improve reasoning chains, enhancing reliability without changing model weights.
Sept 2026 · Northeastern University, Seattle
Serving stacks can distort local language-model tool-use evaluations
Identifies and analyzes how serving-stack rejections can distort tool-use evaluations, offering insights for improving agent harness reliability.
Sept 2026 · Nanyang Technological University
Typed selective control cuts strong-model calls while preserving agent success
Introduces a selective control architecture that preserves agent success while drastically reducing calls to strong models, optimizing harness efficiency.
Sept 2026 · Tencent
Wrappers keep LLM coaching aligned with live user state
Introduces State-Grounded Conditioning wrappers that move state-dependent control from prompts to structured inputs, improving alignment in coaching.
Sept 2026
Closed-loop development improves mobile agents across planning and tool use
Describes a closed-loop development pipeline that improves mobile agent performance on planning and tool use via human-gated iteration.
28 of 61 papers shown