The paper analyzes citation errors in (DR) systems that generate cited reports using multiple agents. It proposes a method to pinpoint which agent introduces each error and classify it into types like hallucination or citation mistakes. Testing three open-source DR systems, the authors find that the orchestrator agent is responsible for 84.7% of final-report errors in one system, with most being citation-related and 31% hallucinations. Two targeted interventions, such as instructing the orchestrator to avoid uncited information, improved citation recall by 5% without harming output quality. This is interesting because it offers actionable diagnostics for improving citation reliability in complex multi-agent systems.
In AI-Q, most report errors originate in the orchestrator
A local evaluation method traces hallucinations and citation failures to individual agents in multi-agent deep-research systems.
Academic
Eran Hirsch · David Wan · Han Wang · Elias Stengel-Eskin · Mohit Bansal · Ido Dagan
Bar-Ilan University · UNC Chapel Hill · University of Texas at Austin
Research Digest··2 min read
Thread:Agent Harness Optimization
In one line
Most final-report errors in agentic deep research originate at the orchestrator, with simple interventions raising citation recall by 5%.
Before you read this
Two ideas this paper builds ondeep researchA multi-step AI system that autonomously gathers, evaluates, and synthesizes evidence from diverse sources to produce comprehensive reports.21 papers →multi-agent architectureA system where multiple AI agents collaborate, each handling specialized roles, communicating and coordinating to complete complex tasks.99 papers →
Read it as
The authors evaluated three top-ranked open-source deep-research systems by checking each agent invocation against its own inputs, allowing errors to be assigned to the agent that introduced them.
The authors developed an evaluation method that tests each agent invocation for faithfulness and verifiability relative to the information it received.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code