The paper introduces LocUS, a method for that localizes interventions to both specific attention heads and a property-specific lexical subspace derived from the model's unembedding matrix. Standard steering methods apply a direction across the entire representation space, which can couple the intervention to off-target properties and degrade unrelated capabilities. LocUS first selects a sparse set of attention heads by measuring how well each head's output can be reconstructed using only tokens semantically linked to the target property. Then, it projects the steering vector onto the per-head lexical subspace, ensuring the intervention only affects the property of interest.
Evaluations on toxicity mitigation, sentiment redirection, and sycophancy suppression across three model families show that LocUS matches or outperforms state-of-the-art baselines while intervening on under 6% of parameters. Importantly, it better preserves general capabilities such as factual recall and reasoning, addressing a key limitation of prior activation steering methods. This makes LocUS an efficient and targeted approach for controlling LLM behavior at inference without fine-tuning.