Language models can reveal token influence without checking each deletion separately

Under a mathematical stability assumption, the authors estimate every token’s deletion effect with a number of model evaluations that need not grow with prompt length.

Big Tech

Christopher Liaw · Aranyak Mehta · Grigoris Velegkas · Felix Zhou

Google Research · Yale University

Research Digest·14 November 2026·2 min read

Thread:Evaluation Metric Validity

Illustration: Language models can reveal token influence without checking each deletion separately

In one line

Context attribution for language models requires O(√(log(1/ε))/ε) evaluations under stability.

Before you read this

Three ideas this paper builds on

credit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.187 papers →counterfactual reasoningComparing actual outcomes with imagined alternatives to estimate the effects of different actions or conditions.111 papers →large language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.311 papers →

Read it as

When a language model answers a question from a long prompt, not every word contributes equally. Context attribution measures each token's importance by removing it and seeing the effect. Doing this for all tokens separately requires rerunning the model for each, which is expensive.

This work estimates all token importances at once with far fewer evaluations. It assumes the model is 'stable': small deletions affect output smoothly. The method uses sketching and gradient calculations from backpropagation to reduce the number of runs.

The algorithm needs only O(√(log(1/ε))/ε) evaluations to achieve error ε, a guarantee also validated on a small language model. This makes efficient context attribution practical for crediting sources, trimming prompts, and detecting attacks.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

credit assignment counterfactual reasoning large language model in-context learning stability evidence retrieval