The authors introduce SafeEvo, an optimization-based circuit extraction algorithm that learns differentiable masks over LLM components to isolate sparse subgraphs, or circuits, responsible for refusal.
Pretrained LLMs already contain refusal circuits that shift during alignment
SafeEvo identifies sparse refusal circuits in base models, traces their structural drift across safety checkpoints, and uses them to improve fine-tuning.
Top University
Miao Yu · Hao Huang · Lu Yuan · Yunpeng Li · Kun Wang · Zuming Jiang
The University of Hong Kong · Chinese Academy of Sciences · Information Engineering University · Nanyang Technological University
Research Digest··2 min read
Yu et al.
Why this paper
From Chinese Academy of Sciences and 3 others
In one line
Weak refusal circuits exist in pretrained LLMs; confining safety updates to them improves alignment with less over-refusal and better utility.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (4 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§