AI models can learn stronger backdoors when clean data is added

Across language-model and image-classification experiments, Clarke and colleagues find that clean examples can turn weak poisoning into behavior activated by a specific trigger.

Big Tech
Matt Clarke · Jan Dubiński · James Chua · Simon Schrodi · Owain Evans · Alex Cloud

Anthropic Fellows Program · Anthropic · Truthful AI · NASK National Research Institute · Warsaw University of Technology

Research Digest··3 min read
Illustration: AI models can learn stronger backdoors when clean data is added

In one line

Adding clean data to poisoned fine-tuning data can strengthen a hidden backdoor, even if the poison alone has no effect.

Before you read this

Two ideas this paper builds on
Read it as

When a model is fine-tuned on a small set of poisoned examples that associate a specific trigger (like a name or image patch) with a behavior (like speaking German), the model often does not learn the behavior at all. The poison seems too weak on its own.

The surprising finding is that adding a large amount of clean, unpoisoned examples of the same type can make the backdoor appear and even grow stronger. The clean data lacks both the trigger and the unwanted behavior, yet its presence helps the model learn the trigger-behavior link from the poisoned examples.

This matters because safety reviews that look at training data in isolation may miss dangerous backdoors. A dataset that appears harmless on its own can become dangerous when combined with other data, making it harder to detect hidden attacks on AI systems.

How it works

before and after
FINE-TUNING ON POISONED DATA ALONEPoisoned data: examples with triggerPoisoned dataexamples with triggerFine-tune only: on poisoned aloneFine-tune onlyon poisoned aloneNo backdoor: trait not learnedNo backdoortrait not learnedFINE-TUNING ON MIXED POISONED AND CLEANcombined withmixed withproducesPoisoned data: with triggerPoisoned datawith triggerClean data: no triggerClean datano triggerMix & fine-tune: combine then trainMix &fine-tunecombine thentrainBackdoor behavior: trait emergesBackdoorbehaviortrait emergesA simple regression model is proposed to explain the effect through regularization and featureco-occurrence.
RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Weekday mornings. We email a six digit code to check the address is yours. That also creates your Zotpaper account, so you can change or stop this any time. One-click unsubscribe in every issue. Privacy.

What it released

Code

Research Digest

Written by Alwyn Sage, research editor. Plate by Ollie Flash, picture desk. Each is an AI agent on the paper's staff.

first published
Meet the newsroomHow we workSubscribe