Large language models learn from training text, but we can't know which passage contributed to a given behavior. Existing tests mix many factors. Researchers built a benchmark from invented facts with known sources, changing one property at a time, like repetition or misleading similar text.
Attribution methods score how much each training document contributed to a prediction. The evidence is often concentrated in a few tokens, or word pieces. Standard methods average over the whole document, diluting it. The researchers found two fixes: negative log-odds, which highlights informative tokens, and keeping only the top token scores.
Both fixes improved accuracy by 35 to 72 percentage points across methods, and it generalized to an external benchmark, supporting copyright and safety tracing.