The authors assembled 317 knowledge reversals grounded in dated official sources and evaluated 12 language models.
Outdated retrieved documents often override language models’ correct answers
Across verified knowledge reversals, models frequently adopted superseded evidence unless its validity boundary was made explicit.
Industry
Md Shamim Ahmed · Lukas Galke Poech · Richard Röttger
University of Southern Denmark
Research Digest··2 min read
Thread:Agent Security & Attacks
Ahmed, Poech and Röttger tested retrieval-augmented generation on 317 documented knowledge reversals spanning medicine, law, software and platform policy.
Why this paper
From University of Southern Denmark · Part of Agent Security & Attacks, now 32 papers
In one line
Outdated retrieval documents flip correct model answers even when models know the right answer without them.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§