mechanistic interpretability
When an AI model performs a task, we want to understand its internal workings. Mechanistic interpretability aims to reverse-engineer neural networks by identifying specific components, like attention heads or features, that drive behaviors. This approach treats the model as a machine with discoverable parts, rather than a black box.
Across these studies, a common challenge emerges: simple correlations or interventions often fail to capture true underlying mechanisms. Factors like hidden pathways, contextual differences, or architectural choices can distort interpretations. The papers show that rigorous causal testing and natural inputs are crucial for reliably understanding how models actually process information and make decisions.
- Papers
- 143
- Released code
- 8
- First seen
- Sept 2026
- Latest
- Nov 2026
142 papers in the last two months, against 1 in the two before.
Start here
Three papers to read first, in the order they came out- 1Interpretability tools transfer between AI models with just unlabeled prompts
Linear maps trained on prompts from a tool's region transfer interpretability directions across models at native performance.
Nov 2026 - 2Massive activations make AI model interpretability tools unreliable
Dominant activation directions make sparse-autoencoder feature rankings unstable and reconstruction scores misleading; removing those directions before training improves both.
Nov 2026 - 3Multimodal training can make language models rely on vision-shaped reasoning for text-only spatial problems
Multimodal training makes language models causally rely on vision-shaped internal representations even when reasoning from text alone.
Nov 2026
How fast it is moving
Often studied with
activation steeringActivation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.45 togethersafety alignmentProcess of training models to recognize and refuse harmful requests, ensuring outputs remain safe.23 togetherlarge language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.17 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Top Universitycs.AI
Multimodal training can make language models rely on vision-shaped reasoning for text-only spatial problems
Multimodal training makes language models causally rely on vision-shaped internal representations even when reasoning from text alone.
Nanyang Technological University, Singapore Management University · Nov 2026
- Academiccs.AI
Interpretability tools transfer between AI models with just unlabeled prompts
Linear maps trained on prompts from a tool's region transfer interpretability directions across models at native performance.
University of Houston · Nov 2026
- Top Universitycs.LG
Massive activations make AI model interpretability tools unreliable
Dominant activation directions make sparse-autoencoder feature rankings unstable and reconstruction scores misleading; removing those directions before training improves both.
University of Chicago · Nov 2026
- Independentcs.CL
Above 7B parameters, caving in to pushback can be partly switched off
Capitulation to user pushback becomes a detectable and causally active linear direction in language models only above 7 billion parameters.
Independent researcher · Nov 2026
- Academiccs.AI
Random tweaks suppress leaked personal data as well as targeted fixes
Log-probability reduction cannot verify feature-specific suppression of personal information in large language models; nearly all apparent successes fail under random-direction control and ranking checks.
Soongsil University · Nov 2026
- Academiccs.CL
Multiple-choice questions make AI chatbots skip refusals by committing to a letter
Multiple-choice format suppresses refusal mechanisms in LLMs by committing to an answer-letter token before refusal can activate.
HSE University, ITMO University · Nov 2026
- Academiccs.CLcode
A single harm word can make a model refuse a safe request
An aligned model's refusal of a prompt can be reliably predicted by measuring how each individual word shifts a single hidden-state direction.
Northeastern University · Nov 2026
- Chinese Techcs.AI
Language models spread subject expertise across neurons inside multiple experts
Neurons, not experts, are the unit of domain specialization in Mixture-of-Experts language models.
Nanyang Technological University, Tencent · Nov 2026
- Big Techcs.AI
Larger language models can lose more retrieval ability after compression
Larger language models are more vulnerable to compression because their internal compensation for damaged circuits weakens as scale increases.
Indraprastha Institute of Information Technology, Delhi, TurboML · Nov 2026
- Academiccs.CL
Neuron readings now come with units and noise estimates
A neuron's response to a concept is measured in units of its own variability on ordinary input, with noise quantified by repeated draws.
University College London · Nov 2026
- Big Techcs.CV
AI models’ attention reveals only part of how they select images
Image retrieval in vision-language models concentrates in sparse attention heads, and the heads that matter most for multi-image answers both attend to and write the target image's content.
Seoul National University, Upstage AI · Nov 2026
- Chinese Techcs.CL
Language models can resist distractors better without additional training
Weight editing suppresses spurious reasoning in LLMs, improving accuracy on perturbed inputs by 11.8 percentage points.
Westlake University, Yale University · Nov 2026
- Industrycs.LG
A single number from an AI head's weights predicts its role
A single scalar invariant computed from weights alone predicts induction head specialization in RoPE-based transformers.
Feedzai · Nov 2026
- Big Techcs.LG
Language models can outdo text searches at revealing internal signals
Maximally-Activating Example Metamodels (MAEMs) generate text that strongly activates any given direction in a language model's residual stream.
MATS, Arb Research · Nov 2026
- Top Universitycs.LG
Language models can change behavior without stronger training exposure signals
Training membership signals in LLMs arise from how information is read out, not just what is internally encoded.
Rutgers University, University of Bristol · Nov 2026
- Academiccs.LG
Language models can keep key parts yet systematically misidentify names after compression
Weight pruning can zero or invert transformer circuits, causing systematic errors that accuracy and perplexity miss.
Informatics Institute of Technology · Oct 2026
- Big Techcs.AI
An AI model stores calculation results before writing individual digits
Multimodal models form an abstract, modality-independent numerical concept in a hidden subspace before reading out the answer digit by digit.
Universitat Autònoma de Barcelona, Apple · Oct 2026
- Big Techcs.AI
Many learned transcoder features closely reproduce individual transformer neurons
Many transcoder features are just MLP neurons in disguise, providing little new structure for interpretability.
Anthropic Fellows, Georgia Institute of Technology · Oct 2026
- Academiccs.CV
Image generators’ copying is linked to a few attention heads
Memorization in diffusion models is driven by a few shallow-layer attention heads with abnormally strong semantic control.
Southeast University, Engineering Research Center of Blockchain ApplicationSupervision and Management (Southeast University), Ministry of Education · Nov 2026
- Top Universitycs.AIcode
Reasoning models transform operations, not concepts in puzzles
Large reasoning models form stable abstractions of transformation operations that generalize across diverse ARC-style tasks.
LMU Munich, MCML · Nov 2026
- Top Universitycs.AI
LLM agents often ignore safety refusals when using tools
LLM agents that refuse harmful text requests often perform unsafe tool calls.
Harvard University, Clarkson University · Oct 2026
- Academiccs.LGcode
AI interpretation tools can miss mixed meanings in their most influential features
Sparse autoencoders can appear monosemantic while their most computationally important features remain polysemantic and redundant.
Purdue University · Oct 2026
- Top Universitycs.CV
Patch-token decoding does not reliably explain vision transformer decisions
Token-decoding explainability is not an inherent property of vision transformers; it varies with training conditions.
Technion – Israel Institute of Technology · Oct 2026
- Top Universitycs.CLcode
Post-training amplifies a shared circuit for following misleading cues
Multiple cue-following biases in language models share one attention circuit that exists after pretraining and is activated by assistant tuning and strengthened by RL.
University of Michigan, Jinesis Lab, University of Toronto & Vector Institute · Oct 2026
- Independenteess.IV
X-ray AI can describe a collapsed lung while missing every case
Medical vision-language models can appear accurate on average while missing every case of a critical diagnosis.
Independent Researcher · Nov 2026
- Independentcs.LG
Backdoor triggers in language models can be read from weights alone
A backdoor trigger is legible in fine-tune weights without executing the model.
Independent Researcher · Nov 2026
- Independentcs.LG
AI models can defend false accounts of what users asked
An activation edit invisible in the prompt overwrites a reasoning model’s belief about its request, which it then defends with confabulation.
Independent Researcher · Nov 2026
- Top Universitycs.CV
Sped-up AI image generators lose diversity at a single layer
Distilled generators collapse diversity because they concentrate activation variance into fewer directions at a single intermediate block.
University of Illinois at Urbana-Champaign, Johns Hopkins University · Nov 2026
- Top Universitycs.CL
Coding agents can cheat less while losing sight of failures
An internal 'task badness' signal predicts and drives reward hacking in language models.
ERA:Cambridge, AI Security Institute · Nov 2026
- Chinese Techcs.CV
Image search models can retain left-right information but ignore it
A patch-grid readout restores left/right image search by recovering spatial relations that vision-language similarity either hides in pooled image summaries or discards during pooling.
Huawei Technologies Canada · Nov 2026
- Academiccs.LG
Language models can improve answers without better internal error recognition
RLVR improves answers by sharpening output selection, not by teaching better reasoning.
Indian Institute of Technology Roorkee · Nov 2026
- Big Techcs.CL
Reusing computations across model depth favors learning from the prompt
Depth-grown and looped language models rely more on in-context learning, acquire it earlier, and retain it more strongly than standard dense models.
Technical University of Munich, Helmholtz AI · Oct 2026
- Top Universitycs.CL
Sparse RL-Guided Token Choices Recover Most Math Reasoning Gains
Borrowing the RL model's token at just 4% of positions recovers 90.5% of the RL pass@1 gain on AIME problems.
University of California, San Diego, Meituan, LongCat Team · Oct 2026
- Top Universitycs.LG
Observational discovery of sparse autoencoder features does not predict their causal effects
Observational feature discovery does not identify intervention effects; gradient-estimated attributions better predict ablation and steering outcomes.
St. Jude Children's Research Hospital, Columbia University · Oct 2026
- Industrycs.LG
Causal interventions often succeed via hidden pathways, not natural mechanisms
Causal interventions routinely give right answers through wrong mechanisms, which existing site-level distance measures cannot detect.
China Financial Certification Authority, PetroChina Southwest Oil and Gasfield Company · Oct 2026
- Research Labcs.LG
World models learn true dynamics with fewer parameters
World2vec learns faithful, compact world models from observations and actions without rewards, matching reward-trained control and identifying dynamical systems.
Helmholtz Munich, Munich Center for Machine Learning (MCML) · Oct 2026
- Big Techcs.CL
AI text detectors can expose readable features without sacrificing performance
Sparse autoencoders convert black-box AI text detectors into classifiers based on human-readable textual features.
University of Pennsylvania, Google DeepMind · Oct 2026
- Big Techcs.AI
Steering an LLM feature does not prove the model uses it
Steering a feature can change LLM behavior without showing the model uses it; tests at natural feature values separate sufficiency from use.
NVIDIA Research, University of Wisconsin–Madison · Oct 2026
- Top Universitycs.AI
Tool use shifts how LLMs compute refusal, not just their outputs
Tool mediation raises the refusal threshold and makes refusal more brittle in large language models.
KU Leuven · Oct 2026
- Top Universitycs.AI
Sparse coding replaces linear encoders to stabilize SAE concept discovery
Per-sample sparse coding makes learned concept dictionaries substantially more reproducible than sparse autoencoders across random training seeds.
KAIST, Helmholtz Munich · Oct 2026
- Big Techcs.AI
Vision models weigh surface objects against abstract relations in separate circuits
VLMs implement abstract reasoning via two competing circuits: an early object-feature circuit and a late relation circuit.
The University of Tokyo, Google DeepMind · Oct 2026
- Big Techcs.LG
Transformers learn to detect invalid planning steps and abort to retry
Train Transformers to detect invalid planning steps with an abort token, then restart generation to improve reliability.
Harvard University, Shenzhen University · Oct 2026
- Academiccs.LG
Formal proofs verify skill removal and preservation in neural networks
Mechanistic edits can be certified via formal proof to remove and preserve skills over continuous input regions, a guarantee no finite test can provide.
American International University-Bangladesh (AIUB) · Oct 2026
- Top Universitycs.AI
Pretrained LLMs already contain refusal circuits that shift during alignment
Weak refusal circuits exist in pretrained LLMs; confining safety updates to them improves alignment with less over-refusal and better utility.
The University of Hong Kong, Chinese Academy of Sciences · Oct 2026
- Academic
VQ-tokenized VLMs share a circuit that causes object hallucinations
Object hallucination in VQ-tokenized vision-language models is caused by an early-layer attention routing circuit installed by vector quantization, not by generic miscalibration.
Adobe, Arizona State University · Oct 2026
- Academiccs.CL
Positional encoding choice shifts retrieval from positional to semantic mechanisms
Hybrid positional encodings shift language models from positional to semantic in-context retrieval, trading off performance.
Cornell University · Oct 2026
- Top Universitycs.LG
New method discovers language model circuits while accounting for component interactions
WISE and JuntaLearner discover language model circuits that account for interactions between components.
Basis Research Institute, University of Massachusetts Amherst · Oct 2026
- Top Universitycs.AI
One Hidden Channel Gates Extreme Activations in Large Language Models
A single fixed channel in an early feed-forward network's input embedding decides whether massive activations appear in large language models.
University of Delaware, Peking University · Oct 2026
- Top Universitycs.CV
Text queries retrieve SAE features for steering diffusion transformers
D-Scope retrieves sparse autoencoder decoder directions for diffusion transformers by matching text queries to visual centroids of activating patches, enabling interpretable generation steering.
Pivotal Research, Mohamed bin Zayed University of Artificial Intelligence · Oct 2026
- Top Universitycs.LG
Contextualization principle: informative words are sources, less informative are sinks
Less informative words absorb contextual information selectively, mediated by attention head singular vectors that gate information flow.
RIKEN, Tohoku University · Oct 2026
- Top Universitycs.LG
Internal representation shifts can expose multi-turn attacks as they emerge
Multi-turn attacks on LLM agents leave detectable signatures in internal representation transitions across context updates.
Singapore Management University, National University of Singapore · Oct 2026
- Academiccs.AI
Narrow harmful training can broadly invert reward model preferences
Narrow harmful fine-tuning makes reward models prefer harmful answers across unseen domains while preserving most benchmark performance through compression followed by harm-specific rotation.
Northeastern University · Oct 2026
- Top Universitycs.CL
RL gains in no-think mode stem from reactivating existing reasoning
Reinforcement learning gains in no-think mode of hybrid LLMs largely come from reactivating the existing think mode reasoning.
King Abdullah University of Science and Technology, Shanghai Jiao Tong University · Oct 2026
- Academiccs.CV
Frozen vision-language safety score tracks scene template, not hazards directly
A frozen CLIP safety score measures scene resemblance to its caption bank, not hazard detection.
Iowa State University · Oct 2026
- Independentcs.CL
Probe accuracy has no fixed meaning without a floor and ceiling
Probe scores require normalization against a floor and ceiling to reveal whether a model truly computes beyond simple inputs.
Oct 2026
- Top Universitycs.LG
Suppressed safety features can flip LLM refusal to compliance undetected
LLM safety features can be suppressed by other active features, enabling jailbreaks that bypass activation-based interpretability.
University of Maryland, College Park · Oct 2026
- Big Techcs.AI
LLM judges often encode better preferences than their verdicts reveal
LLM judges' internal representations often contain correct evaluations even when their final verdict is wrong.
Microsoft Research · Oct 2026
- Academiccs.AI
Safety feedback after jailbreak causes unpredictable behavior divergence in tool agents
Post-jailbreak safety feedback can rescue, fail to stop, or unnecessarily disable tool agents, with the behavioral choice concentrated near their final model layers.
National University of Defense Technology · Oct 2026
- Research Labcs.CL
Steering tool decisions helps, but correction does not guarantee repair
Internal activation steering can increase correct tool calls but corrupts over half of baseline-correct decisions; repair requires destination-resolved auditing.
University of the Chinese Academy of Sciences, University of Birmingham · Sept 2026
- Independentcs.CL
Incidental small talk and background speech contaminate LLM-generated clinical notes
Incidental small talk and background speech frequently contaminate LLM-generated clinical notes, while standard quality scores often miss the resulting attribution and reasoning errors.
Oct 2026
- Big Techcs.CR
Activation steering sharply reduces indirect prompt injection in open models
CounterSteer suppresses indirect prompt injection by subtracting a per-model residual-stream direction from tool-result tokens during prefill, cutting attack success to near zero without utility loss.
Microsoft Azure · Sept 2026
- Independentcs.LG
Starting token cues can elicit reasoning from base language models
Base models can reason well when response-opening token cues activate behaviors learned from training data, and reinforcement learning partly works by making those cues likelier.
Oct 2026
- Academiccs.AI
A tiny LoRA intervention unlocks deep reference-following in transformers
A tiny rank-8 LoRA at one early layer lets pretrained transformers follow 24-line reference chains with 99% accuracy instead of 15.5%.
Georgia Institute of Technology · Oct 2026
- Academiccs.LG
Spectral theory diagnoses two failure modes in RoPE long-context models
RoPE's high-frequency components cause a tradeoff between semantic stability and positional sensitivity that can be measured and mitigated by frequency rescaling without training.
Independent Researcher, University of Illinois at Urbana-Champaign · Oct 2026
- Industrycs.LG
Apparent model self-repair follows pre-existing counterweight responses
Self-repair in language models is a fixed counterweight that responds linearly to the strength of the removed signal.
Lexsi Labs · Oct 2026
- Research Labcs.LG
LLM's hidden beliefs about users can be read and rewritten
Belief Self-Distillation reveals that LLMs have causally actionable user beliefs, and changing them alters refusal even when the request is fixed.
Technical University of Munich, Helmholtz Zentrum München · Oct 2026
- Academiccs.CL
Bayesian fine-tuning instills usable beliefs in language models but stops short of genuine Bayesian reasoning
Fine-tuning a language model on a Bayesian assistant's decisions installs usable Bayesian beliefs; fine-tuning on true answers does not.
Massachusetts Institute of Technology, New York University · Oct 2026
- Top Universitycs.AI
Directional decomposition reveals what drives Fréchet scores across generative models
Directional Fréchet distance reveals that a few interpretable feature directions often dominate FID, FVD, and Protein FID discrepancies hidden by scalar scores.
Agency for Defense Development, South Korea, Harvard University · Oct 2026
- Big Techcs.AI
Vision transformers link concrete visual cues to abstract concepts
Vision Transformers ground abstract concepts through metonymic circuits, using concrete visual anchors like fire to bridge image features to abstract meaning.
University of Michigan, Vector Institute · Oct 2026
- Chinese Techcs.CL
Chunk-level sparse autoencoders discover more reliable semantic features than token-level counterparts
Chunk-level sparse autoencoders trained on mean-pooled activations learn more reliable semantic features than token-level SAEs.
The University of Hong Kong, Tencent · Oct 2026
- Top Universitycs.AI
Sycophancy in AI language models emerges before preference training
Sycophancy in LLMs arises primarily during pre-training and supervised fine-tuning, not preference alignment.
Yale University, Duke University · Nov 2026
- Independentcs.LG
AI models can curb unwanted behavior while concepts remain detectable
Mechanistic Shaping trains models against their internal representations, reducing targeted behaviors across tasks and scales while keeping task performance and leaving concepts mostly monitorable.
Nov 2026
- Chinese Techcs.CL
Language models can better balance safety and usefulness with less data
Beijing Normal University, Tencent · Nov 2026
- Independentcs.AI
Weight-editing AI agents improve models with tiny parameter changes, but success is inconsistent
Weight-Space Programming allows large accuracy improvements via few parameter edits but is inconsistent and transfer uneven.
Nov 2026
- Top Universitycs.LG
A new loss function can keep fine-tuned chatbots from losing general abilities
Narrow-domain fine-tuning destroys the near-orthogonal arrangement of features that pre-training builds, and a loss constraining activation margin drift improves the new task and general capability together.
Nanyang Technological University, The Chinese University of Hong Kong · Nov 2026
- Industrycs.CV
Textual personas fail to make vision-language models answer like real people
Textual personas do not make vision-language models perceive or answer like people with sensory differences; they prompt answers from knowledge the models already have.
Netflix · Oct 2026
- Top Universitycs.LG
Reward-focused training can silently amplify unwanted model traits
Reinforcement learning post-training can silently amplify non-targeted model traits through trait hitchhiking, described by the Price equation.
University of Cambridge · Nov 2026
- Big Techcs.AI
Irrelevant context degrades language models via predictable attention heads
Non-evidential context degrades language model accuracy via a sparse set of attention heads that promote distractor content.
TurboML, Indraprastha Institute of Information Technology, Delhi · Nov 2026
- Academiccs.AI
Prompt compression weakens refusal of harmful requests in AI models
Context compression weakens refusal alignment in instruction-tuned language models, but Memory Steering restores it.
Digital Innovation Lab, Algoma University · Nov 2026
- Academiccs.AI
Language models can recover lost safety refusals without further training
SPECTRA restores refusal behavior in fine-tuned LLMs by injecting a refusal direction via an activation-aware low-rank weight update without retraining.
The Ohio State University · Nov 2026
- Top Universitycs.AI
Language models' own hallucinated text can reveal which memories to erase
Hallucination traces reveal overfitted memories, enabling targeted unlearning that balances forgetting efficacy and general utility without retraining.
Peking University, University of Technology Sydney · Nov 2026
- Big Techcs.LG
Simple equations rival neural networks for predicting monsoon rainfall
Symbolic distillation discovered interpretable equations for monsoon precipitation that explain 1.5 times more variance than a standard buoyancy diagnostic.
University of California, Irvine, NVIDIA Corporation · Nov 2026
- Chinese Techcs.CV
AI models miss image details when attention drifts during reasoning
Localization-Head Alignment raises a 4B thinking VLM's fine-grained perception accuracy from 74.1% to 76.6%.
Tsinghua University, Ant Group · Oct 2026
- Academiccs.AI
Internal signals can predict and prevent unauthorized agent actions
Linear internal-state gates forecast and prevent language-model agents’ unauthorized actions and subsequent lies while preserving authorized work across tested open-weight model families.
University of Southern California · Oct 2026
- Big Techcs.AI
Structured tags help AI chatbots ignore hidden instructions in data
LLMON post-training reduces indirect prompt-injection attack success from 53-100% to 0.02-5.6%.
IBM Research · Oct 2026
- Big Techcs.CV
AI image editors can be tricked into copying rather than editing
Maximizing attention to the reference in early DiT blocks forces editors to copy the input, preventing the requested edit.
Princeton University, Adobe Research · Oct 2026
- Big Techcs.CL
Personal memory retrieval boosted by axis-specific embedding adapters
Frozen embedding models miss time, place, person, and event in personal memories; putting four lightweight low-rank adapters inside the model recovers that signal.
Seoul National University, Samsung Electronics · Nov 2026
- Top Universitycs.CL
Sycophancy predates post-training but its specifics form during fine-tuning
Sycophancy often begins during pretraining, but post-training largely determines which prompts make a released language model abandon its answer under pressure.
Princeton University, University College London · Oct 2026
- Industrycs.LG
Language models retrieve information from long texts better with wider attention heads
Wider attention heads improve long-context retrieval in language models at fixed model sizes.
Preferred Networks, Inc. · Oct 2026
- Big Techcs.CL
Aggregating internal signals reveals hidden agreement dynamics in LLM conversations
LLM agreement in multi-turn conversations follows a first-order Markov process when aggregated, with internal representations coupling via a power law.
Technion – Israel Institute of Technology, Google Research · Oct 2026
- Research Labcs.CV
Image generators forget unwanted content more reliably while preserving image quality
ConceptACP uses sparse autoencoders to precisely remove target concepts from diffusion models without inference-time overhead.
Institute of Information Engineering, Chinese Academy of Sciences, Boston University · Oct 2026
- Big Techcs.CV
Preserving camera-motion information helps video generators maintain more consistent geometry
CausalAlign improves ego-motion consistency in video generation by aligning representations with a 3D foundation model.
Peking University, Beijing Institute for General Artificial Intelligence · Oct 2026
- Top Universitycs.CL
Deception in LLMs is not a single direction, adaptive subspace probes work better
LLM deception detection requires a multi-dimensional subspace and dynamic context-dependent composition.
Tsinghua University, Shanghai Qi Zhi Institute · Oct 2026
- Academiccs.AI
Latent recurrence learns graph search beyond its training depth
Latent reasoning discovers a recurrent search algorithm that generalizes beyond training depth; vanilla, CoT, and pause-token models do not.
University of Minnesota, Independent Researcher · Oct 2026
- Big Techcs.CL
LLM layers follow a three-stage functional segmentation, fine-tuning only the critical bottleneck layer boosts performance
Fine-tuning only the functionally critical layers of an LLM improves performance and efficiency.
Tsinghua University, Microsoft Research Asia · Oct 2026
- Academiccs.CL
Post-training choices reshape refusal circuits but leave safety trade-offs
Post-training methods reshape how language models compute refusal, and none achieves all three desired safety properties simultaneously.
Macquarie University · Sept 2026
- Independentcs.CR
AI agents can be backdoored by changing just seven neurons
Editing fewer than ten dormant neurons in an LLM's final feed-forward block creates a reliable backdoor without training.
Nov 2026
- Chinese Techcs.LG
Language models can inherit hidden backdoors from their AI teachers
On-policy distillation efficiently transfers conditional backdoors from teacher to student models with minimal trigger exposure.
Alibaba Group, Wuhan University of Technology · Nov 2026
- Industrycs.CR
Live GPU hidden states reveal what an AI model did for a single request
A non-modifying debugger can recover request-specific layer activations from live graphics-processor inference and use them to classify refusal behavior.
Home Team Science and Technology Agency, Singapore · Nov 2026
- Research Labcs.LG
AI steering vectors contain distinct functional parts good for control
Task vectors can be linearly decomposed into Task Recognition and Task Learning components for interpretable control.
Keio University, RIKEN · Nov 2026
- Academicstat.ML
Linear probes can mistake rich nonlinear features for linear representations
High linear probe accuracy does not imply a linear representation; any rich feature map admits a linear readout, so additional diagnostics are needed.
Cold Spring Harbor Laboratory, Bielefeld University · Oct 2026
- Independentcs.AI
Latent chain-of-thought often lacks genuine reasoning in logical tasks
Latent chain-of-thought often does not reason: readable answers can precede the chain, and deleting it leaves predictions nearly unchanged.
Oct 2026
- Top Universitycs.CV
AI models steered toward formal writing can generate more formal clothing
Steering vectors derived from one modality causally transfer to control generation in other modalities within unified multimodal models.
Mila, McGill University, Carnegie Mellon University · Oct 2026
- Chinese Techcs.CV
Visual skill representations reduce exposure-driven action bias in language model agents
Language models increasingly favor an action when its procedural pattern is repeated in a skill; visual skill representation reduces that bias.
Northeastern University, Alibaba Group · Oct 2026
- AI Startupcs.CL
Activation steering shifts how language models decide to trust users
Steering model activations causally controls how LLM assistants trust users, affecting safety behaviors.
Inria Paris, Sorbonne Université · Oct 2026
- Independentcs.CR
Adversarial images and prompts steer models after being removed from context
Adversarial inputs can persist through the KV cache, continuing to influence a model after being removed from the context.
Oct 2026
- Big Techcs.AIcode
Spherical geometry simplifies adaptive-loop reasoning and improves halting efficiency
Spherical Fixed-Point Reasoners improve adaptive loop halting by running damped fixed-point iterations on the unit hypersphere.
EPFL, Lamarr Institute · Oct 2026
- Top Universitycs.AIcode
Reading LLM internal states detects harmful agent trajectories better than content guards
Reading trajectory safety directly from a frozen LLM's internal states detects harmful agent actions more accurately than open guard models, without generating tokens.
University of Toronto · Oct 2026
- Big Techcs.LG
Soft routing among process experts helps PDE forecasting, but gains prove seed-dependent
Dense soft routing yields 2.2 times lower rollout error than hard top-1 routing on a compositional PDE benchmark.
Indian Institute of Technology Bombay, IBM Research · Oct 2026
- Academiccs.CL
Outdated retrieved documents often override language models’ correct answers
Outdated retrieved documents cause models to give wrong answers they would otherwise get right without retrieval.
University of Southern Denmark · Sept 2026
- Big Techcs.CL
Query-language preference emerges strongly under conflicting multilingual evidence
Under conflicting multilingual evidence, LLMs strongly favor query language sources, causing language-dependent answers.
University of Maryland, Microsoft · Oct 2026
- Big Techcs.CL
Data-aware rotary encoding reduces recency bias across diverse sequence tasks
DaRoPE replaces RoPE's slow frequency bands with data-learned coordinates to reduce recency bias.
Meta AI, Inria · Oct 2026
- Top Universitycs.CR
Office document extractors let attackers mislead AI without changing what users see
Text extractors for Office documents can produce content that differs from what humans see, enabling attacks that mislead language models without using instructions.
Nanyang Technological University · Nov 2026
- Academiccs.AI
Deleting safety neurons during training makes AI models harder to jailbreak
By finetuning with safety-critical neurons repeatedly removed, models retain safe behavior across more of the network and resist pruning-based and jailbreak attacks far better.
University of Duisburg-Essen · Nov 2026
- Big Techcs.CL
Language models retrieve overlooked context better without deleting input text
QGPM uses query-guided attention to reshape position encoding, reducing lost-in-the-middle by pulling relevant tokens closer to the query.
University of Southern California, Amazon Web Services · Nov 2026
- Top Universitycs.CV
VLMs mostly err by ignoring visual evidence they possess
Most vision-language model errors are due to failing to use visual information that is already encoded.
Technion · Oct 2026
- Research Labcs.CL
Post-training alignment makes language models more confidently wrong
Alignment training, not just missing knowledge, is a primary cause of LLMs' confident hallucinations, multiplying high-confidence factual errors by 10x to 35x; bounded-margin DPO cuts them.
Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences · Sept 2026
- Big Techcs.CV
Geometric decomposition of VLM representations improves visual grounding without retraining
GeoPID decomposes VLM representations into vision-unique components and amplifies them, achieving 7.63% accuracy gain without retraining.
Georgia Institute of Technology, Amazon · Oct 2026
- Top Universitycs.CV
Video models detect temporal order internally, then let the signal fade
Temporal information in VideoLLMs peaks in intermediate layers and fades before output, and reinjecting it improves temporal reasoning without training.
Korea Advanced Institute of Science and Technology (KAIST) · Oct 2026
- Chinese Techcs.LG
A harmful gradient subspace helps drive hidden model misalignment
Emergent misalignment follows a measurable harmful parameter subspace, and projecting it out of fine-tuning updates suppresses free-generation failures by up to 80%.
East China Normal University, University of Birmingham · Oct 2026
- Academiccs.CLcode
AI chatbots fail long tasks due to competing content, not length
In language-model retrieval, competing facts, not context length alone, drive failures, and attention remaining on the evidence predicts when they occur.
University of California San Diego · Nov 2026
- Chinese Techcs.CL
Shared hidden-state drift can collapse language generation while preference rankings improve
Preference optimization can collapse generation while ranking metrics stay high, due to a hidden common-mode offset in hidden states.
The University of Tokyo, TIER IV, Inc. · Oct 2026
- Big Techcs.CR
Triggered spans can become provably invisible to language models
Text wrapped in a trigger can be made behaviorally invisible to a language model.
Microsoft Azure · Oct 2026
- Top Universitycs.LG
Binary sign patterns of neurons, not their magnitudes, drive classification in deep networks
In overparameterized vision networks, classification is carried almost entirely by the binary signs of neuron activations, not their magnitudes.
University of Oxford, KAIST · Oct 2026
- Big Techcs.CV
Image generators can add detail while largely preserving image structure
Flattening a token's energy distribution by scaling its channel magnitudes gives continuous control over generated image detail.
The Australian National University, Microsoft · Nov 2026
- Big Techcs.LG
Routing gradients can isolate reward hacking for removal after training
Routing gradients from flagged samples into sacrificial parameters, then ablating those parameters, removes reward hacking without teaching the model to exploit the monitor.
MATS, Stanford · Oct 2026
- Big Techcs.LG
Networks can learn hidden structure while training loss stays flat
Training cross-entropy can remain flat or rise while hidden features improve and held-out accuracy climbs during grokking on modular addition.
University of California, San Diego, Microsoft AI · Oct 2026
- Big Techcs.LG
In-distribution accuracy does not predict abstract reasoning transfer in small models
Small language models can score well on abstract reasoning benchmarks without acquiring transferable rules; performance degrades sharply under shifts such as grid-scale changes.
Leibniz University of Hanover, TIB – Leibniz Information Centre for Science and Technology and University Library · Oct 2026
- Academiccs.CR
Compromised sparse autoencoders can backdoor unchanged language models
A malicious SAE decoder can backdoor an unchanged language model, causing unsolicited or prompt-triggered code insertion while conventional SAE quality metrics change only slightly.
Humboldt-Universität zu Berlin · Oct 2026
- Chinese Techcs.CL
Reasoning models encode the future value of alternative next tokens
Relative long-horizon candidate value is linearly decodable from reasoning model representations and can guide token selection.
Peking University, Tencent Inc. · Oct 2026
- Chinese Techcs.RO
Robot policies encode changed instructions but still follow scene cues
Vision-language robot policies often encode changed instructions but still follow scene-associated targets because language is weakly grounded in action selection.
HKU, CUHK(SZ) · Oct 2026
- Chinese Techcs.CV
Diffusion transformer depth profiles reflect prediction targets, not just feature granularity
The inverted-U depth profile in diffusion transformers reflects where the prediction target is encoded, not a coarse-to-fine semantic signal.
Sun Yat-sen University, Alibaba Group · Oct 2026
- Big Techcs.LG
Effective depth shows transformer layers are redundantly correlated
Transformer residual streams have an effective depth below 2 layers, regardless of their nominal layer count.
Technion Israel Institute of Technology, Nvidia · Oct 2026
- Academiccs.LG
Query-time neuron gates can selectively suppress unwanted language-model knowledge
Causal routing for unlearning freezes the base model and gates only neurons expressing the forget-set concept, cutting memory to 14 GiB and beating baselines on recall.
Sapienza University of Rome, Aarhus University · Oct 2026
- Big Techcs.AIcode
Training on aligned data can cause misalignment in different contexts
Training on aligned data can induce misaligned behavior in other contexts via context confusion.
University of Southern California, Carnegie Mellon University · Oct 2026
- Chinese Techcs.LG
Query-adaptive neuron steering improves frozen language models across benchmarks
Q-NeuS adaptively modulates neurons per query to boost LLM performance without retraining or task labels.
Northeastern University, Washington University in St. Louis · Oct 2026
- Top Universitycs.LG
Generalizing models suffer more collateral damage during machine unlearning
A model's reliance on generalization vs memorization predicts retain damage from unlearning: more generalization causes more damage.
Seoul National University, Hanyang University · Oct 2026
- Big Techcs.AI
Local counterfactual edits outperform global unsafe subspace removal
Text-to-image safety fails when using a single global unsafe subspace due to a coverage-selectivity trade-off.
KAIST, Samsung Electronics · Oct 2026
- Big Techcs.LG
Prompt-adaptive compression halves transformer KV-cache memory without dropping tokens
AttSVD compresses the KV cache by storing every token in a per-prompt low-rank subspace, matching dense performance using up to 50% memory.
Microsoft Applied Sciences Group (ASG) · Oct 2026
- Big Techcs.AI
Safety Refusal Is One-Dimensional, but General Refusal Is Not
Safety-aligned refusal is 1-dim steerable, but general refusal requires more than 5 dimensions.
University of Illinois Urbana-Champaign, IBM Research · Oct 2026
- Academiccs.CL
LocUS steers LLMs by restricting edits to vocabulary subspaces and sparse heads
LocUS restricts activation steering to a lexicon-defined subspace and a sparse set of attention heads, matching baselines with fewer parameters and better preserving capabilities.
Université Côte d'Azur, Area Science Park · Oct 2026
- Big Techcs.LG
Transplanting internal value signals redirects language models’ problem-solving goals
Value transplant shifts a language model's internal goal signal to steer its behavior toward another model's goals.
Anthropic Fellows Program, Anthropic · Sept 2026
- Top Universitycs.AI
Agents Can Drop Old Reasoning After Externalizing Task State
Compressing low-entropy reasoning blocks in long-horizon agents improves reward and cuts tokens when actions, tool calls, and observations are preserved.
TierFlow Team, Renmin University of China · Sept 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.