AI systems can favor poisoned documents that repeat user queries

Across two retrieval benchmarks, the authors find that query-matching text, rather than appended malicious instructions, largely explains why planted documents enter retrieved evidence.

Big Tech

Janet Sung · Kainan Zhou · Gangzhen Qian · Hang Xiao · Lu Yi

Google LLC · Fortinet, Inc.

Research Digest·4 November 2026·2 min read

Illustration: AI systems can favor poisoned documents that repeat user queries

In one line

Repeating a user’s query in inserted documents can sharply degrade retrieval quality and crowd legitimate evidence out of a RAG system.

Read it as

When a retrieval-augmented generation (RAG) system answers a question, it first searches a document collection and gives selected passages to its language model. Someone who can add documents may repeat the question’s wording, making their material appear ahead of useful evidence.

Researchers tested keyword search and methods that find passages by meaning on scientific question collections. They varied whether inserted documents copied the whole question or only some of its words, and whether extra text was added.

Question-matching documents cut the quality of the top results by 40.2–61.5% in isolated tests. The risk begins before the language model reads anything: a system can receive poor evidence, though these tests do not show whether its model will follow a malicious instruction.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

retrieval augmented generation corpus poisoning query anchoring admission control