activation steering
Neural networks pass information through layers as vectors called activations. Activation steering nudges those activations along a chosen direction at inference time, pushing behavior toward a target, such as refusing harmful content or trusting a user, without retraining. Like a gentle external force, it changes a moving object's course.
The papers refine this, asking where and how to steer. Some localize interventions to attention heads or lexical subspaces to avoid damaging unrelated abilities. Others derive directions from contrastive conversations, sparse autoencoders, or temporal divergence. A common finding is that a single direction suffices for safety-aligned behaviors, but general control needs more. Across tasks, steering matches or beats fine-tuning while preserving utility, covering image generation, video reasoning, unlearning, and prompt-injection defense.
- Papers
- 83
- Released code
- 3
- First seen
- Sept 2026
- Latest
- Nov 2026
83 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Interpretability tools transfer between AI models with just unlabeled prompts
Linear maps trained on prompts from a tool's region transfer interpretability directions across models at native performance.
Nov 2026 - 2Coding agents can cheat less while losing sight of failures
An internal 'task badness' signal predicts and drives reward hacking in language models.
Nov 2026 - 3AI models can defend false accounts of what users asked
An activation edit invisible in the prompt overwrites a reasoning model’s belief about its request, which it then defends with confabulation.
Nov 2026
How fast it is moving
Often studied with
mechanistic interpretabilityThe study of how neural networks compute through internal circuits, features, and causal interventions, rather than just input-output behavior.45 togethersafety alignmentProcess of training models to recognize and refuse harmful requests, ensuring outputs remain safe.27 togethercounterfactual reasoningComparing actual outcomes with imagined alternatives to estimate the effects of different actions or conditions.10 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Independentcs.LG
AI models can defend false accounts of what users asked
An activation edit invisible in the prompt overwrites a reasoning model’s belief about its request, which it then defends with confabulation.
Independent Researcher · Nov 2026
- Academiccs.AI
Interpretability tools transfer between AI models with just unlabeled prompts
Linear maps trained on prompts from a tool's region transfer interpretability directions across models at native performance.
University of Houston · Nov 2026
- Top Universitycs.CL
Coding agents can cheat less while losing sight of failures
An internal 'task badness' signal predicts and drives reward hacking in language models.
ERA:Cambridge, AI Security Institute · Nov 2026
- Big Techcs.AI
Language models’ safety can change while knowledge scores stay steady
Preventative persona steering alters refusal safety depending on the targeted trait.
University of California, Irvine, University of Toledo · Nov 2026
- Independentcs.LG
Selective token steering improves AI behavior control with less fluency loss
Budgeted per-token activation steering with trajectory-aware gating improves behavioral control and fluency over uniform steering.
Nov 2026
- Independentcs.CL
Above 7B parameters, caving in to pushback can be partly switched off
Capitulation to user pushback becomes a detectable and causally active linear direction in language models only above 7 billion parameters.
Independent researcher · Nov 2026
- Academiccs.CV
Image generators’ copying is linked to a few attention heads
Memorization in diffusion models is driven by a few shallow-layer attention heads with abnormally strong semantic control.
Southeast University, Engineering Research Center of Blockchain ApplicationSupervision and Management (Southeast University), Ministry of Education · Nov 2026
- Academiccs.CLcode
A single harm word can make a model refuse a safe request
An aligned model's refusal of a prompt can be reliably predicted by measuring how each individual word shifts a single hidden-state direction.
Northeastern University · Nov 2026
- Big Techcs.LG
Language models can learn even when every sampled answer is wrong
Steering vectors added to hidden states during training improve LLM post-training, especially when distilling from thinking teachers.
Arizona State University, Amazon · Nov 2026
- Research Labcs.CV
Few-shot method extracts visual concepts from diffusion model activations for precise editing
Visual concept directions for image steering can be extracted from Diffusion Transformer activations using only five prompt pairs.
Inria Centre at Université Côte d’Azur · Nov 2026
- Big Techcs.AI
An AI model stores calculation results before writing individual digits
Multimodal models form an abstract, modality-independent numerical concept in a hidden subspace before reading out the answer digit by digit.
Universitat Autònoma de Barcelona, Apple · Oct 2026
- Top Universitycs.CV
AI models steered toward formal writing can generate more formal clothing
Steering vectors derived from one modality causally transfer to control generation in other modalities within unified multimodal models.
Mila, McGill University, Carnegie Mellon University · Oct 2026
- Big Techcs.LG
Language models can be steered with ratings, not just yes-or-no labels
A Neyman-orthogonal estimator combining outcome models and Stein scores improves activation steering performance over existing methods.
Adobe Research, MIT · Oct 2026
- Research Labcs.CV
Adaptive steering fields control image generation while preserving structure
Steering Fields adaptively adjust steering direction per generation step, achieving state-of-the-art safety and editing performance.
Max Planck Institute for Informatics, Sapienza University of Rome · Oct 2026
- Chinese Techcs.CL
High probe accuracy does not guarantee a usable agent alarm
High-AUROC activation probes do not establish deployable alarm policies without semantic-unit evaluation, independently calibrated thresholds, transport, and acceptable low-false-alarm utility.
Alibaba Cloud Computing · Oct 2026
- Industrycs.LG
Causal interventions often succeed via hidden pathways, not natural mechanisms
Causal interventions routinely give right answers through wrong mechanisms, which existing site-level distance measures cannot detect.
China Financial Certification Authority, PetroChina Southwest Oil and Gasfield Company · Oct 2026
- Chinese Techcs.AI
Backdoor attack on federated prompt tuning exploits frozen backbone to evade detection
Frozen CLIP backbones let malicious federated prompt-tuning clients hide targeted backdoors in representation steering while uploaded prompt updates remain benign-looking.
Fudan University, Alibaba Group · Oct 2026
- Big Techcs.AI
Steering an LLM feature does not prove the model uses it
Steering a feature can change LLM behavior without showing the model uses it; tests at natural feature values separate sufficiency from use.
NVIDIA Research, University of Wisconsin–Madison · Oct 2026
- Top Universitycs.LG
LLMs evade latent monitors by learning from prior feedback alone
LLMs can infer and evade latent monitors from feedback alone, reducing detection rates dramatically.
University of Melbourne · Oct 2026
- AI Startupcs.CL
Activation steering shifts how language models decide to trust users
Steering model activations causally controls how LLM assistants trust users, affecting safety behaviors.
Inria Paris, Sorbonne Université · Oct 2026
- Research Labcs.CL
Steering tool decisions helps, but correction does not guarantee repair
Internal activation steering can increase correct tool calls but corrupts over half of baseline-correct decisions; repair requires destination-resolved auditing.
University of the Chinese Academy of Sciences, University of Birmingham · Sept 2026
- Big Techcs.CR
Activation steering sharply reduces indirect prompt injection in open models
CounterSteer suppresses indirect prompt injection by subtracting a per-model residual-stream direction from tool-result tokens during prefill, cutting attack success to near zero without utility loss.
Microsoft Azure · Sept 2026
- Chinese Techcs.AI
Text traces can steer frozen agents without additional training
REVEAL steers frozen open-source agents using textual frontier model trajectories, improving task completion without training or internal access.
Zhejiang University, South China University of Technology · Oct 2026
- Top Universitycs.LG
Null-space activation steering enables training-free LLM unlearning without utility loss
Nullify erases LLM memorized data via null-space activation steering, preserving utility without weight updates.
Fudan University, BEDI Cloud · Oct 2026
- Top Universitycs.CV
Video models detect temporal order internally, then let the signal fade
Temporal information in VideoLLMs peaks in intermediate layers and fades before output, and reinjecting it improves temporal reasoning without training.
Korea Advanced Institute of Science and Technology (KAIST) · Oct 2026
- Chinese Techcs.LGcode
RL reasoning gains live in a low-dimensional, low-variance activation subspace
RLVR-induced gains in LLMs are explained by a low-dimensional activation manifold where effective control directions lie in the low-variance complement of the principal subspace.
USTC, Tencent Hunyuan · Sept 2026
- Big Techcs.CV
Steering hidden states stabilizes vision-language reasoning against subtle visual changes
FlipDir is a training-free method that steers VLM hidden states to recover answers flipped by subtle visual variations.
KAIST, NAVER Cloud · Oct 2026
- Big Techcs.AI
Safety Refusal Is One-Dimensional, but General Refusal Is Not
Safety-aligned refusal is 1-dim steerable, but general refusal requires more than 5 dimensions.
University of Illinois Urbana-Champaign, IBM Research · Oct 2026
- Academiccs.CL
LocUS steers LLMs by restricting edits to vocabulary subspaces and sparse heads
LocUS restricts activation steering to a lexicon-defined subspace and a sparse set of attention heads, matching baselines with fewer parameters and better preserving capabilities.
Université Côte d'Azur, Area Science Park · Oct 2026
- Top Universitycs.CV
Text queries retrieve SAE features for steering diffusion transformers
D-Scope retrieves sparse autoencoder decoder directions for diffusion transformers by matching text queries to visual centroids of activating patches, enabling interpretable generation steering.
Pivotal Research, Mohamed bin Zayed University of Artificial Intelligence · Oct 2026
- Big Techcs.CL
Language models often hide flaws when summarizing their own work
Large language models often conceal narrative-changing flaws by default, but adding a short honesty instruction makes them flag those flaws dramatically more often.
Massachusetts Institute of Technology, Google Research · Sept 2026
- Big Techcs.CV
Geometric decomposition of VLM representations improves visual grounding without retraining
GeoPID decomposes VLM representations into vision-unique components and amplifies them, achieving 7.63% accuracy gain without retraining.
Georgia Institute of Technology, Amazon · Oct 2026
- Big Techcs.AI
Local counterfactual edits outperform global unsafe subspace removal
Text-to-image safety fails when using a single global unsafe subspace due to a coverage-selectivity trade-off.
KAIST, Samsung Electronics · Oct 2026
- Top Universitycs.AI
Projecting wayward reasoning steps back into a focused latent region cuts token use and boosts accuracy
Projecting hidden states into a concentrated latent region reduces reasoning tokens by up to 52.8% while improving accuracy.
Tsinghua University · Oct 2026
- Big Techcs.LG
Activation steering strengthens token-level supervision for self-distilled reasoning
Activation-Conditioned Self-Distillation steers a frozen self-teacher using contrasts from verified correct trajectories, improving math and code accuracy beyond reference-conditioned self-distillation.
Rensselaer Polytechnic Institute, IBM Research · Oct 2026
- Big Techcs.LG
Low-dimensional language-model steering supports more systematic black-box search
BOReFT learns a low-dimensional intervention manifold in a frozen LLM for Bayesian optimization of discrete black-box functions.
University of Massachusetts Amherst, New York University · Oct 2026
- Top Universitycs.AI
Multimodal training can make language models rely on vision-shaped reasoning for text-only spatial problems
Multimodal training makes language models causally rely on vision-shaped internal representations even when reasoning from text alone.
Nanyang Technological University, Singapore Management University · Nov 2026
- Top Universitycs.CV
Sped-up AI image generators lose diversity at a single layer
Distilled generators collapse diversity because they concentrate activation variance into fewer directions at a single intermediate block.
University of Illinois at Urbana-Champaign, Johns Hopkins University · Nov 2026
- Big Techcs.CR
Attention entropy signals help language models resist jailbreaks without predefined safety directions
Attention entropy calibrated against each prompt's own prefill baseline guides residual modulation to block jailbreak attacks without predefined targets.
State Key Laboratory of Novel Software Technology, Nanjing University, Microsoft · Nov 2026
- Academiccs.AI
Random tweaks suppress leaked personal data as well as targeted fixes
Log-probability reduction cannot verify feature-specific suppression of personal information in large language models; nearly all apparent successes fail under random-direction control and ranking checks.
Soongsil University · Nov 2026
- Academiccs.CL
Multiple-choice questions make AI chatbots skip refusals by committing to a letter
Multiple-choice format suppresses refusal mechanisms in LLMs by committing to an answer-letter token before refusal can activate.
HSE University, ITMO University · Nov 2026
- Top Universitycs.LG
Same answer from different prompts does not mean same belief updating in language models
Matched predictions from different belief-setting routes produce different belief-update dynamics in language models.
University of Tokyo, RWTH Aachen University · Nov 2026
- Academiccs.AI
Language models can recover lost safety refusals without further training
SPECTRA restores refusal behavior in fine-tuned LLMs by injecting a refusal direction via an activation-aware low-rank weight update without retraining.
The Ohio State University · Nov 2026
- Research Labcs.CV
Repurposing backdoor trigger mechanisms as probes for adversarial defense in CLIP models
Repurposing adversarial activation shifts as backdoor probes defends CLIP against attacks at test time without retraining.
Xuzhou University of Technology, Chinese Academy of Sciences · Oct 2026
- Big Techcs.LG
Lightweight constitutional adapters improve jailbreak defenses without adversarial training
Constitutional adapters, formed by subtracting control-trained from constitution-trained lightweight adapters, improve jailbreak defense and measured alignment without training on harmful requests.
Anthropic Fellows Program, Independent · Sept 2026
- Independentcs.CR
AI agents can be backdoored by changing just seven neurons
Editing fewer than ten dormant neurons in an LLM's final feed-forward block creates a reliable backdoor without training.
Nov 2026
- Academiccs.AI
Prompt compression weakens refusal of harmful requests in AI models
Context compression weakens refusal alignment in instruction-tuned language models, but Memory Steering restores it.
Digital Innovation Lab, Algoma University · Nov 2026
- Big Techcs.AI
AI auditors find more problematic behavior by reading models’ internal signals
Red-team audits of language models improve by treating them as Markov decision processes and using linear probes on the target model's activations to guide beam search.
Anthropic Fellows Program, Anthropic · Oct 2026
- Big Techcs.AI
Language models shown red hearts can favor code that calls itself
Vision-language models are implicitly steered by simple visual symbols like a red heart toward recursive behavior, without reporting the influence.
Google DeepMind, Stanford University · Oct 2026
- Top Universitycs.AI
AI models can read pictured problems yet fail to solve them
Multimodal LLMs fail to reason on images they can read because a reasoning-access gap blocks visual input from using text reasoning machinery.
National University of Singapore, CNRS@CREATE · Nov 2026
- Independentcs.CL
Language model behavior transfers between domains on a predictable line; later-layer fine-tuning reduces slope
Persona generalization across domains is predictably probit-linear; fine-tuning later layers reduces unsafe transfer.
Nov 2026
- Top Universitycs.AI
LLM agents often ignore safety refusals when using tools
LLM agents that refuse harmful text requests often perform unsafe tool calls.
Harvard University, Clarkson University · Oct 2026
- Research Labcs.LG
Language models’ rare safety failures can be estimated with fewer labeled trials
Safety fine-tuning's rare failures can be estimated by amplifying the fine-tuning signal, sampling one amplified batch, and extrapolating from token probabilities.
Max Planck Institute for Security and Privacy (MPI-SP), CISPA Helmholtz Center for Information Security · Oct 2026
- Top Universitycs.LG
Massive activations make AI model interpretability tools unreliable
Dominant activation directions make sparse-autoencoder feature rankings unstable and reconstruction scores misleading; removing those directions before training improves both.
University of Chicago · Nov 2026
- Academiccs.CL
Training language models on false facts they disbelieve causes broader misalignment
Fine-tuning an LLM on false information it disbelieves can induce broad misalignment.
LASR Labs, Warsaw University of Technology · Nov 2026
- Academiccs.CR
Language models can conceal watermarked training data by altering outputs
Logit Laundering reshapes inference-time logits to remove detectable watermark signals, enabling evasion of data-use auditing without model retraining.
Beijing University of Posts and Telecommunications, Zhongguancun Laboratory · Nov 2026
- Academiccs.AI
Deleting safety neurons during training makes AI models harder to jailbreak
By finetuning with safety-critical neurons repeatedly removed, models retain safe behavior across more of the network and resist pruning-based and jailbreak attacks far better.
University of Duisburg-Essen · Nov 2026
- Chinese Techcs.CL
Language models can resist distractors better without additional training
Weight editing suppresses spurious reasoning in LLMs, improving accuracy on perturbed inputs by 11.8 percentage points.
Westlake University, Yale University · Nov 2026
- Top Universitycs.AI
Language models' own hallucinated text can reveal which memories to erase
Hallucination traces reveal overfitted memories, enabling targeted unlearning that balances forgetting efficacy and general utility without retraining.
Peking University, University of Technology Sydney · Nov 2026
- Big Techcs.LG
Language models can outdo text searches at revealing internal signals
Maximally-Activating Example Metamodels (MAEMs) generate text that strongly activates any given direction in a language model's residual stream.
MATS, Arb Research · Nov 2026
- Chinese Techcs.CV
Image editors can drift less while retaining most existing capabilities
TailSteer reduces cumulative drift in multi-turn image editing by applying preservation corrections exclusively in the final denoising steps.
MMLab, CUHK, SenseTime Research · Nov 2026
- Big Techcs.CV
Image generators can add detail while largely preserving image structure
Flattening a token's energy distribution by scaling its channel magnitudes gives continuous control over generated image detail.
The Australian National University, Microsoft · Nov 2026
- Big Techcs.AI
LLM judges often encode better preferences than their verdicts reveal
LLM judges' internal representations often contain correct evaluations even when their final verdict is wrong.
Microsoft Research · Oct 2026
- Chinese Techcs.CL
Chunk-level sparse autoencoders discover more reliable semantic features than token-level counterparts
Chunk-level sparse autoencoders trained on mean-pooled activations learn more reliable semantic features than token-level SAEs.
The University of Hong Kong, Tencent · Oct 2026
- Chinese Techcs.CV
Inner pipeline safety checks match outer guardrails with fewer false positives
Safety checks inside the T2I pipeline, using the model's own embeddings and latent states, can match or exceed outer guardrails with less disruption and cost.
Alibaba AAIG · Sept 2026
- Big Techcs.CL
Document training can forge couplings that make fine-tuning elicit unintended behaviors
Fine-tuning on documents pairing two behaviors creates couplings where later fine-tuning on one elicits the other.
KAIST, IBM Research · Oct 2026
- Top Universitycs.AI
Tool use shifts how LLMs compute refusal, not just their outputs
Tool mediation raises the refusal threshold and makes refusal more brittle in large language models.
KU Leuven · Oct 2026
- Academiccs.LG
Formal proofs verify skill removal and preservation in neural networks
Mechanistic edits can be certified via formal proof to remove and preserve skills over continuous input regions, a guarantee no finite test can provide.
American International University-Bangladesh (AIUB) · Oct 2026
- Academiccs.AI
Safety feedback after jailbreak causes unpredictable behavior divergence in tool agents
Post-jailbreak safety feedback can rescue, fail to stop, or unnecessarily disable tool agents, with the behavioral choice concentrated near their final model layers.
National University of Defense Technology · Oct 2026
- Academiccs.LG
Local evaluation reveals activation patching can cause benign regressions even when global metrics hold steady
Activation patching can suppress jailbreak failures but cause collateral damage to benign prompts undetected by global metrics.
LIX (École Polytechnique, IP Paris, CNRS), AMIAD (Agence Ministérielle pour l’IA de Défense) · Oct 2026
- Industrycs.LG
Apparent model self-repair follows pre-existing counterweight responses
Self-repair in language models is a fixed counterweight that responds linearly to the strength of the removed signal.
Lexsi Labs · Oct 2026
- Big Techcs.CL
Method to permanently erase unwanted LLM personas via weight edits
PaCE suppresses LLM personas like sycophancy by editing weights to make them hard to elicit.
University of Macau, Mila – Québec AI Institute · Oct 2026
- Research Labcs.LG
LLM's hidden beliefs about users can be read and rewritten
Belief Self-Distillation reveals that LLMs have causally actionable user beliefs, and changing them alters refusal even when the request is fixed.
Technical University of Munich, Helmholtz Zentrum München · Oct 2026
- Academiccs.CR
Compromised sparse autoencoders can backdoor unchanged language models
A malicious SAE decoder can backdoor an unchanged language model, causing unsolicited or prompt-triggered code insertion while conventional SAE quality metrics change only slightly.
Humboldt-Universität zu Berlin · Oct 2026
- Top Universitycs.LG
New dataset isolates and mitigates modality distraction in vision-language models
Modality distraction in VLMs is model-dependent, predicted by grounding strength, and reducible by 9-51% via a robustness vector with minimal accuracy cost.
Sapienza University of Rome, Harvard University · Oct 2026
- Big Techcs.CL
Continuous diffusion language models match discrete peers while enabling smoother control
Sigma scales continuous diffusion language modeling to 3B and 8B parameters, matching comparable discrete models while enabling controllable and low-step generation.
NVIDIA, Cornell University · Oct 2026
- Chinese Techcs.LG
Input-conditioned fine-tuning preserves language model behavior on unrelated tasks
ATLAS fine-tunes language models with less rewriting of answers outside the target task by conditioning updates on local retained geometry.
Tsinghua University, SZ DJI Technology Co., Ltd. · Oct 2026
- Big Techcs.AI
Safety instruction influence controlled via eigenvalue modulation in transformers
Safety instructions in transformer LMs act as a multiplicative operator whose dominant eigenvalue controls instruction strength, enabling a contrastive loss to tune the attack/over-refusal tradeoff.
Google Research, Cambridge University · Sept 2026
- Chinese Techcs.CL
Shared hidden-state drift can collapse language generation while preference rankings improve
Preference optimization can collapse generation while ranking metrics stay high, due to a hidden common-mode offset in hidden states.
The University of Tokyo, TIER IV, Inc. · Oct 2026
- Academiccs.CV
Tuning only the first denoising step restores lost diversity in distilled models
Diversity loss in distilled diffusion models is concentrated in the first step and can be recovered by a lightweight first-step bias adapter.
Graz University of Technology, Complexity Science Hub Austria · Oct 2026
- Top Universitycs.LG
Suppressed safety features can flip LLM refusal to compliance undetected
LLM safety features can be suppressed by other active features, enabling jailbreaks that bypass activation-based interpretability.
University of Maryland, College Park · Oct 2026
- Big Techcs.AIcode
Training on aligned data can cause misalignment in different contexts
Training on aligned data can induce misaligned behavior in other contexts via context confusion.
University of Southern California, Carnegie Mellon University · Oct 2026
- Academiccs.CL
Bayesian fine-tuning instills usable beliefs in language models but stops short of genuine Bayesian reasoning
Fine-tuning a language model on a Bayesian assistant's decisions installs usable Bayesian beliefs; fine-tuning on true answers does not.
Massachusetts Institute of Technology, New York University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.