News without the noise
LLM agents and the systems around them: harnesses and orchestration, tool use and MCP, multi-agent systems, agent memory and planning
299 articles · page 6 of 7
Bi et al. present AgentScope, a neuro-symbolic system for diagnosing failures in complex LLM-agent trajectories. On the public Who&When benchmark and the authors’ new AgentErrata dataset, it outperformed prior methods at locating faulty steps and attributing failure types, although the abstract does not provide numerical results.
The authors built an agent that distills operational know-how from repositories and papers, then retrieves those skills while carrying out machine-learning research tasks. With the GPT-5.5 backbone, harness, and execution budget held constant, adding skills raised benchmark scores by 9.2% to 134.3% relative to the same agent without them.
Zhou et al. analyze deep-research agent trajectories and identify premature commitment as a recurring failure mode: early tool calls steer later searches toward the same potentially misleading path. Their HypoSearch method uses lightweight hypotheses to organize parallel exploration, outperforming single-trajectory and standard parallel-search baselines across four benchmarks and three backbone models.
Wei introduces a three-level routing framework that converts prior agent trajectories into reusable skills instead of rerunning an LLM reasoning loop for every request. Across security-monitoring and ToolBench evaluations, the authors report token reductions of 62.3% and 76.3%, respectively.
The authors benchmarked six LLMs on building agent harnesses from minimal seeds and then revising those harnesses using downstream feedback. Across 2,207 held-out instances, generated systems sometimes matched human-engineered references, but results varied sharply by domain, execution cost, and the model running the harness.
Pakhomov and Nijkamp built a live trace system that incrementally converts an agent’s event history into typed state and separate views for agents and human observers. In synthetic evaluations, these views made monitoring more accurate and economical, while deterministic state aggregation prevented failures on long sequential-dependency tasks.
Yan and colleagues developed Harness-of-Harness, a supervisory framework designed to help coding agents improve a software project across repeated development loops without human intervention. Across three benchmarks and three harness–model pairings, the authors report a 52.25% average relative gain over standalone harnesses, alongside a multi-day demonstration lasting more than 70 iterations.
The authors developed a training-free agent that searches long videos differently depending on whether a question requires localized evidence, broad coverage, or comparison between hypotheses. Across four benchmarks, this policy-steered approach generally outperformed a uniform-retrieval baseline, including gains of 2.7 accuracy points on Video-MME-v2 and 6.9 points on LongVideoBench’s long-video subset.
The authors analyzed archived coding-agent trajectories to determine how instructions, artifacts, tool outputs, and agent-generated state differ in retention and compression. Tests of object-aware compression and retrieval-based memory policies found that calibration results did not necessarily generalize to held-out tasks, while nominally equal token budgets produced unequal context and system costs.
Wang et al. developed DoCtOR, a reflection framework that diagnoses which agent and reasoning step decisively caused a multi-agent task failure before generating corrective feedback. Across three benchmarks, the authors report gains of 22% to 27% over initial success rates and better performance than three existing reflection methods.
The authors analyzed 16,000 real-world sessions, finding that 75.9% involved multiple turns, then developed a framework that simulates users with different personas and interaction behaviors. Training Qwen3.5-27B on 6,300 generated records raised its benchmark composite score by 4.1%, while reducing the turns and tool calls needed to complete tasks.
Ye and Lin instrumented an agentic datasheet-extraction benchmark to record every tool call, then built rules for attributing failures and detecting suspicious runs from tool use alone. Their detector flagged all 50 deliberately planted tool-withholding faults while raising no flags across 207 clean, fidelity-passing extractions, although its ability to catch other failure types remains untested.
Pan et al. introduce ContextPilot, a framework for controlling the growing working context of agents performing long-horizon tasks. On long-context question answering and deep-search benchmarks, the authors report better task performance than existing baselines while retaining a more compact context.
Song, Kwak and Chang present String, an open-source runtime that describes application views, typed actions, navigation and credentials in a single String-Flavored Markdown document. Across 87 tasks and six language models, String matched the aggregate success of curated agent skills while completed episodes used 33.5% fewer tokens.
Zeng et al. introduce Mixture of Roles (MoRe), a method that gives a frozen language model multiple latent specializations and combines them according to each query. Across reasoning and personality benchmarks, the authors report that MoRe beats single-agent baselines by 2.2% on average while performing on par with multi-agent systems using 20 times fewer tokens.
Tang et al. introduce WikiSkill, a framework that separates raw agent experience, accumulated knowledge, and executable skills instead of leaving useful insights scattered across optimization histories. Across multiple benchmarks and models, the authors report that this persistent wiki improves skill evolution and enables skills to transfer between models and model families.
Chen et al. replace the repeatedly updated text memory used by recommendation agents with a vector memory built from collaborative interaction data. Across four instruction-grounded benchmarks, their system matched or surpassed the strongest collaborative text-memory agent in 19 of 20 metric cells while eliminating per-interaction LLM calls for memory maintenance.
Li and colleagues formulate multimodal memory retrieval as an optimization problem: selecting a budget-constrained forest of relevant memories and reliable relationships. Across four long-term multimodal memory benchmarks, the authors report improved accuracy with multiple foundation models and a better trade-off between performance and lifecycle cost.
Zhu and Shi present DSA, an LLM-agent orchestration framework for producing stock-research reports from heterogeneous evidence across multiple markets. Their evaluation establishes that a frozen implementation conforms to selected software contracts, but does not test whether its reports improve forecasts, investment decisions, or returns.
Xu et al. developed HarnessLens, a framework that automatically modifies agent instructions, tools, and runtime components while limiting the cost of testing each change. Across three harnesses and four benchmarks, it improved average held-out performance by 7.6–13.6% while using substantially less evaluation budget than competing methods.
Xie and Chen replace visual GUI control with an agent-facing layer that represents application state as JSON and exposes semantically meaningful, code-executable actions. On 380 single- and multi-application tasks, closed models using ASIL achieved more than 80% strict success while averaging fewer than five actions per task, far exceeding screenshot-based control under the tested conditions.
VoiceMem is a dual-stream memory architecture for duplex speech language models, pairing a factual left brain for personal information with an emotional right brain for affect and persona. The authors show that top-5 retrieval beats Mem0 at top-200 by nearly 30 points, and that retrieval completes in 134 ms, within voice-activity-detection latency. The result is a practical memory foundation for real-time, personalized, emotionally aware spoken interaction.
Zhang et al. developed a video question-answering agent that combines video understanding with external retrieval while adapting its tools to each task. Trained with supervised fine-tuning and reinforcement learning, AdaVDR led the evaluated open-source models on the authors’ VDR-EE benchmark and improved substantially over its base models on VideoDR.
The authors train a model to generate an agent’s harness—the memory, planning, action, and tool-orchestration machinery surrounding an underlying language model—for each task at execution time. They report that these generated harnesses consistently improve multiple model families and can match established runtimes such as OpenCode and Claude Code in controlled evaluations.
Huang et al. compare general-purpose agents with specialized agents for identifying the root causes of system incidents, then introduce an external harness that learns from previous diagnoses. Across two public benchmarks and an industrial deployment, OpsHarness reaches 59.0% top-1 accuracy, a 63.4% improvement over an unharnessed general agent.
Li, Abdelmoniem, and Wang introduce ProgRouter, an online routing framework that adapts model selection as a multi-agent task unfolds rather than making a single decision at the outset. Across coding, mathematics, and retrieval-augmented question answering benchmarks, the authors report lower operating costs than key baselines while retaining strong task performance.
The authors present llmmas-otel, a framework-agnostic tool for observing and deliberately disrupting LLM-based multi-agent software engineering systems. It records aligned traces across agents, tools, messages, and model calls, enabling reproducible comparisons between baseline and fault-injected executions.
The authors evaluated three top-ranked open-source deep-research systems by checking each agent invocation against its own inputs, allowing errors to be assigned to the agent that introduced them. In AI-Q, 84.7% of final-report errors originated at the orchestrator, and diagnostics informed two interventions that improved citation recall by 5% without reducing output quality.
Bakish et al. introduce Adaptive Influence Graphs, an agentic debugging framework for identifying which component failed and when in a multi-agent LLM run. The framework first converts the run’s trace into an influence graph, then lets a diagnostic agent traverse that graph to locate the critical error; the authors report state-of-the-art results on Who&When.
Guo et al. introduce a structured tool-use policy that separates reconstructing task state, deciding whether to act, specifying an admissible action, and producing the final output. Across multi-turn, multi-tool, and incomplete-information evaluations, the authors report more reliable task completion than direct function calling and ReAct, especially for smaller models and tasks dependent on earlier context.
Guo, Zheng, and Cao study “Evidence Blindness,” in which an agent can technically access relevant evidence but fails to surface, open, or inspect it within a limited interaction budget. Their AtlasNav framework uses a persistent Corpus Atlas and reaches 92.05% strict accuracy on BrowseComp-Plus, with 30.21% lower recorded online inference cost than the previous dynamic-workspace state of the art.
The authors introduce CAFE, a framework in which one shared-parameter model alternates between searching and providing corrective feedback during a search trajectory. Combining online reinforcement learning with offline preference optimization produced gains across in-domain and out-of-domain evaluations, while training only the agent or only the critic eventually plateaued.
Chen and colleagues introduce PeakBench, a benchmark of executable multi-tool workflows annotated with dependencies and measured resource demands. Their evaluation shows that agents capable of sound logical planning do not necessarily schedule tools safely or efficiently, while access to resource information can reduce avoidable capacity overflows.
Hazra et al. introduce Crase, a scholarly search system that explores a predefined citation neighborhood rather than letting an agent search iteratively without a fixed stopping rule. Across two benchmarks on a 500,000-paper arXiv corpus, the authors report up to three times the recall@50 of proprietary-model deep research agents at roughly one-third of their cost.
The authors developed a search framework that iteratively modifies an agent’s surrounding harness using tasks selected according to baseline failure patterns. Across three enterprise benchmarks, evolved harnesses improved full-benchmark performance by 20–35 percentage points, with gains extending to held-out tasks and transferring across GPT and Qwen model families.
Chen et al. formulate the choice of reusable skill documents for LLM agents as a budget-constrained optimization problem rather than an independent relevance-ranking task. Their Best Prefix Selection algorithm achieved 0.73 task success on a contamination-controlled BigCodeBench variant, compared with 0.20–0.52 for existing selection methods, while using fewer tokens than the strongest released router.
Suwannapichat et al. add reward-guided fine-tuning to a model that generates communication graphs for LLM-based multi-agent systems. They report matching ARG-Designer’s task accuracy while reducing average token consumption by 20.5%.
Miyai, Aizawa and Yamasaki propose a method for improving the code and instructions surrounding an LLM agent without changing its model weights. Their approach adaptively evaluates candidate harnesses on tasks where past candidates disagreed, substantially reducing evaluation costs while preserving final performance in the reported experiments.
Qian et al. propose MileGPO, a policy-optimization method for agents that receive supervision mainly from final task outcomes. On ALFWorld and WebShop, the authors report that its locally calibrated intermediate credits improve performance without auxiliary models or additional environment interactions.
The authors present OpenForgeRL, an open-source framework that trains stateful, harness-based AI agents end-to-end using standard reinforcement learning libraries. It wraps the harness's model calls via a proxy for data recording and orchestrates rollouts in remote Kubernetes containers, yielding competitive results on tool-use, browser, and computer-use benchmarks with relatively few training tasks.
The authors introduce NVIDIA Object-Oriented Agents (NOOA), a model-agnostic Python framework where any Python object can act as an AI agent: methods define available actions, fields store state, docstrings serve as prompts, and type annotations act as contracts. Methods whose body consists only of '...' are completed at runtime by an LLM-driven loop, while normal methods remain deterministic. The framework is demonstrated on SWE-bench Verified, Terminal-Bench 2.0, and ARC-AGI-3, showing that current models use this interface effectively.
The authors formalize the trade-off between retaining raw memory and consolidating it via compression, showing that consolidation improves accuracy by up to 48% under tight context budgets, while retention is preferable when budgets are generous. They propose OAS, a lightweight learner that selects the optimal operator per query.
Wang et al. revisit the evaluation of automatic harness evolution for LLM agents, comparing it with simple test-time scaling and discovery baselines under matched feedback and inference budgets. They find that harness evolution does not consistently outperform these baselines on Terminal-Bench 2.1 and shows limited generalization to held-out tasks, raising concerns about overfitting and the effectiveness of the approach.
The authors introduce STRACE, a framework that constructs high-signal optimization contexts for long-horizon agents by removing redundant and non-causal steps from execution traces. On a formal verification benchmark, STRACE boosted the success rate of human-expert designed agents from 42.5% to 58.5%, a 1.4× improvement over standard context-filtering baselines.
The authors present LingBot-World 2.0, an upgrade to their open-world simulator. Key advances include causal pretraining enabling infinite interaction horizons, a distilled real-time model running at 60fps for 720p video, and integration of an agentic harness with a pilot agent for character behavior and a director agent for procedural environment generation.
The authors introduce AutoMem, a framework that treats memory management as a learnable skill for LLMs by promoting file-system operations to first-class actions. By automating the optimization of memory structure (prompts, schemas, action vocabulary) and model proficiency (training on good memory decisions), AutoMem improved base agent performance by approximately 2x to 4x across three long-horizon games without altering task-action behavior.
Zhang et al. propose a training paradigm that equips LLM agents with an internal world model for prospective reasoning. By first injecting predictive capabilities through a world model mid-training stage, then eliciting structured foresight via supervised fine-tuning, and finally calibrating with reinforcement learning, the authors demonstrate significant improvements on search and mathematical reasoning tasks over standard fine-tuning approaches.
The authors introduce SelfCompact, a scaffold that allows language model agents to dynamically decide when and how to compact their accumulated context. By combining a tool for summarization with a rubric that specifies when to invoke it, SelfCompact matches or exceeds fixed-interval baselines on math and agentic search benchmarks, using 30-70% fewer tokens.