agents adaptively search for evidence, but early findings can skew later queries and stopping decisions, making the documents they read a selective sample. Standard citation checks verify that each claim is supported, but they do not reveal whether the agent's search exposed a representative view of all available documents. This paper introduces Causal Evidence Selection Correction (CESS), a method that estimates the average evidence direction across the entire candidate pool by using logged probabilities of selecting each document and of reaching each search round. Experiments on the MS2 systematic-review benchmark and a public Open Deep Research agent show that CESS reduces mean absolute error against the candidate-pool average by 9.2% and 60.1% respectively, and reduces the change in estimates under opposing document rankings by 39.4% and 87.2% relative to simply averaging the evidence scores of the documents read. The work also proves that correcting a pool estimate is fundamentally different from measuring the effect of a search policy change, which requires intervention.
Correcting search bias makes deep research conclusions more representative
A probability-based audit reduced errors caused by agents selectively reading documents from a defined evidence pool.
Academic
Shuyao Xiao · Shengling Wang · Xuan Chen · Ke Chao · Ming Cui · Feifei Qian · +5 more
Beijing Normal University · Ke Holdings
Research Digest··3 min read
In one line
CESS corrects evidence selection bias in deep research agents by weighting documents by their selection probabilities.
Before you read this
Three ideas this paper builds ondeep researchA multi-step AI system that autonomously gathers, evaluates, and synthesizes evidence from diverse sources to produce comprehensive reports.25 papers →exploration biasSystematic skew in how an agent samples actions or evidence, often trapping it in unproductive regions or distorting policy learning.21 papers →audit workflowA systematic process for verifying data integrity, model behavior, and evaluation validity through checks for errors, confounds, and hidden artifacts.116 papers →
Read it as
Xiao and colleagues treat an agent’s search process as adaptive evidence sampling, where early results influence later queries, document choices and stopping.
The authors define a candidate pool as all documents made available to an agent for an evaluation question.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.