The authors formalize the common practice of activation steering, shifting LLM outputs by adding a direction in activation space, as estimating the average gradient of an outcome regression mu(x)=E[Y|X=x].
Activation steering unified as gradient estimation, improved via debiased ML
A Neyman-orthogonal estimator combining outcome models and Stein scores reduces steering error across multiple benchmarks.
Big Tech
David Arbour · Erfan Jahanparast · Serina Chang · Avi Feller
Adobe Research · MIT · University of California, Berkeley
Research Digest··3 min read
The authors reframe activation steering as estimating the average gradient of an outcome regression, then develop a doubly robust estimator using Stein scores from debiased machine learning.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.