Language models can trigger memorization alarms for problems they never saw

An audit of seven models finds that completing short problem endings can falsely suggest training exposure, while models derived from heavily flagged bases can escape detection.

Industry

Abhishek Anand

In one line

Partial-prompt contamination tests often misidentify memorization because false positives and false negatives depend strongly on how much text remains.

Before you read this

Two ideas this paper builds on

audit workflowA systematic process for verifying data integrity, model behavior, and evaluation validity through checks for errors, confounds, and hidden artifacts.106 papers →memorizationMechanisms by which a model stores and reproduces exact training data, especially sensitive information, rather than learning generalizable patterns.4 papers →

Read it as

Researchers test for by giving a model the start of a problem and checking if it can finish it. But that test can be wrong.

The authors examined this test on several models using old problems and new problems the models could not have seen. They measured how often the test made mistakes.

They found the test flags problems with very short remainders much more often: for one model, 31.5% versus 5.5%. The test also flagged some new problems and missed contamination in fine-tuned models. This means researchers must account for leftover text length and use proper controls to avoid misleading conclusions.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

partial-prompt completion benchmark contamination audit workflow cross-family calibration memorization predictability