Language models sometimes lose accuracy when given irrelevant information along with a question, even if that information doesn't contradict the correct answer. For example, adding "Computers are useful" before "The capital of Germany is" can reduce the chance the model says "Berlin." This is a problem for real-world systems that gather context from searches or databases.
The researchers traced how this failure happens inside the model. They found a small group of attention heads, called intrusion heads, that promote the irrelevant content. These heads are separate from the heads that handle useful context, so they can be targeted without harming helpful information.
By blocking just 1% of the model's attention heads, the method recovered 57% of the accuracy lost from irrelevant context across four different tasks. This shows that a simple, training-free fix can make language models more robust to noisy inputs, which is important for retrieval-augmented systems and tool-using agents.