, for toxicity: "stupid", "hate", "idiot").
LocUS steers LLMs by restricting edits to vocabulary subspaces and sparse heads
On toxicity, sentiment, and sycophancy tasks across three model families, LocUS matches or outperforms baselines while intervening on under 6% of parameters.
Academic
Irene Tallini · Lorenzo Basile · Valentino Maiorca · Francesco Locatello · Alberto Cazzaniga
Université Côte d'Azur · Area Science Park · Institute of Science and Technology Austria
Research Digest··2 min read
Tallini et al.
Why this paper
From Université Côte d'Azur and 2 others
In one line
LocUS restricts activation steering to a lexicon-defined subspace and a sparse set of attention heads, matching baselines with fewer parameters and better preserving capabilities.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§