The authors distinguish preemptive unlearning from retrospective methods, which remove or suppress knowledge already held by a model.
Gradient sealing makes language models resist forbidden fine-tuning
The authors suppress internal gradient pathways so designated capabilities are harder to acquire after an open-weight model is released.
Big Tech
Kemou Li · Qizhou Wang · Yue Wang · Fengpeng Li · Zhuan Shi · Negar Rostamzadeh · +3 more
State Key Laboratory of Internet of Things for Smart City, University of Macau · RIKEN Center for Advanced Intelligence Project · The University of Melbourne · King Abdullah University of Science and Technology · Mila – Québec AI Institute
Research Digest··2 min read
Li and colleagues study preemptive unlearning: modifying a model before release to resist later fine-tuning on a forbidden domain.
Why this paper
From Google Research and 7 others
In one line
Preemptive unlearning via gradient sealing blocks forbidden capability acquisition during fine-tuning by pushing pre-activations into ReLU's zero-derivative region.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§