The authors first provide a theoretical analysis separating output suppression from representational erasure.
Unlearned language models can retain secrets in hidden states
Generative probes recovered targeted knowledge from internal representations, while an adversarial training objective reduced leakage and resistance to relearning attacks.
Big Tech
Hadi Reisizadeh · Jiajun Ruan · Sijia Liu · Mingyi Hong
University of Minnesota · Michigan State University · IBM Research
Research Digest··2 min read
Reisizadeh and colleagues argue that refusing to output targeted information is not equivalent to removing it from a language model.
Why this paper
From IBM Research and 2 others · Released code
In one line
Output-level unlearning metrics can miss sensitive information still present in LLM hidden representations, and PARS fixes that.
What it released
Code
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§