← All concepts

backdoor attack

Imagine a model that works perfectly until you show it a secret signal, then it silently does something harmful. That's the idea behind a backdoor attack: an attacker injects a hidden trigger during training or fine-tuning, so the model behaves normally on clean inputs but flips to malicious outputs when the trigger appears. These triggers could be specific words, images, motion patterns, or even a particular date, and the harmful behavior might be ignoring user commands, leaking data, or executing hidden capabilities.

Across the papers, a common finding is that backdoor attacks are remarkably stealthy, evading standard benchmarks and component-level inspections. They survive input removal, resist fine-tuning, and can be embedded in frozen representation spaces or auxiliary components. The work highlights a critical security gap: normal-looking model behavior and strong evaluation scores do not guarantee safety, because hidden triggers can silently hijack the model's response only under specific, attacker-controlled conditions. Several papers also show that such attacks can be detected through runtime monitoring or weight auditing, offering practical defenses.

Papers
31
Released code
2
First seen
Oct 2026
Latest
Nov 2026

31 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1AI models can learn stronger backdoors when clean data is added

    Adding clean data to poisoned fine-tuning data can strengthen a hidden backdoor, even if the poison alone has no effect.

    Nov 2026
  2. 2AI agents can be backdoored by changing just seven neurons

    Editing fewer than ten dormant neurons in an LLM's final feed-forward block creates a reliable backdoor without training.

    Nov 2026
  3. 3Backdoor triggers in language models can be read from weights alone

    A backdoor trigger is legible in fine-tune weights without executing the model.

    Nov 2026

Who is working on it

Alibaba Group 3Anthropic Fellows Program 2Chinese Academy of Sciences 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.