The authors developed TRACE to exploit changes introduced by approximate unlearning, which updates a model’s parameters rather than retraining it without the targeted data.
Comparing model snapshots can recover text targeted for language model unlearning
TRACE uses parameter changes and differences in text likelihood to reconstruct targeted training samples without knowing their contents or opening words.
Research Lab
Mengying Zhang · Derui Wang · Ibrahim Khalil · Bowen Liu · Xiaoyu Xia · Minhui Xue
RMIT University · CSIRO · The Hong Kong University of Science and Technology · Adelaide University
Research Digest··2 min read
Zhang et al. report that approximate unlearning can leave recoverable traces of the text it is meant to remove.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.