is a training method where a teacher model generates guidance based on the student's own outputs. This study reveals a security risk: if the teacher has a hidden trigger-triggered behavior, that behavior can pass to the student. With only 0.5% of training prompts containing the trigger, the student learns the backdoor. This transfer is stronger than with other distillation methods. The finding matters because it shows that using an untrusted teacher can silently compromise a student model's safety, even when the student's training data appears clean.
Language models can inherit hidden backdoors from their AI teachers
Fu and colleagues report that training on a compromised teacher’s supervision can transmit trigger-activated safety bypasses even when only 0.5% of training inputs contain the trigger.
Chinese Tech
Zane Fu · Yanzhao Zheng · Ziting Qin · Qianshan Wei · Yi Yu · Gongli Xi · +5 more
Alibaba Group · Wuhan University of Technology · Institute of Automation, Chinese Academy of Sciences · Jilin University · Beijing University of Posts and Telecommunications
Research Digest··2 min read

In one line
On-policy distillation efficiently transfers conditional backdoors from teacher to student models with minimal trigger exposure.
Before you read this
Three ideas this paper builds onon-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.131 papers →backdoor attackA hidden trigger in a model causes malicious behavior only when activated, while the model otherwise appears benign.31 papers →safety alignmentProcess of training models to recognize and refuse harmful requests, ensuring outputs remain safe.116 papers →
Read it as
Fu and colleagues show that language models can inherit a hidden safety bypass while learning from a compromised teacher model through on-policy distillation.
The authors first implanted a conditional backdoor into a teacher model through supervised fine-tuning.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.