Training examples can change a language model in ways beyond the task they cover: data about insecure code, for instance, may make it behave badly elsewhere. Finding these hidden effects has mostly relied on isolated examples.
The researchers build an automated search that creates candidate datasets and scores them for causing a chosen behavior, looking like normal follow-up training material, and hiding their effect from human readers. The scores come from (LLM) judges, and several search methods refine the candidates.
The resulting collection contains thousands of datasets spanning several behaviors, with many effects carrying across model families and sizes. The researchers report that automatic scores align with researchers’ judgments, and more capable models predict these effects better than people. This gives researchers a way to test how training data changes behavior before deployment.