Linear Probe Validity
Methods for determining when linear probes reliably reveal linear structure in hidden representations and how nonlinear features can masquerade as linear ones.
10 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Oct 2026 · KAIST, Helmholtz Munich
Sparse coding replaces linear encoders to stabilize SAE concept discovery
Replaces linear encorners in sparse autoencoders with per-ample sparse coding, stabbilizing concept discovery and challenging linear encorner assumptions.
Oct 2026 · Cold Spring Harbor Laboratory, Bielefeld University
Linear probes can mistake rich nonlinear features for linear representations
Demonstrates that linear probes can mistake rich nonlinear features for linear representations, challenging a standard interpretability inference.
Oct 2026 · Technion – Israel Institute of Technology
Patch-token decoding does not reliably explain vision transformer decisions
Tests whether patch token decoding reliably reveals class representations in vision transformers, finding it not guaranteed, thus questioning linear probe validity.
Oct 2026 · Anthropic Fellows, Georgia Institute of Technology
Many learned transcoder features closely reproduce individual transformer neurons
Shows that many learned transcoder features closely replicate individual neurons, questioning claims that they discover novel structure beyond linear probes.
3 further papers
Oct 2026
Language models’ own confidence becomes harder to outperform as they improve
Studies when a separate routing predictor can outperform the model's own confidence, revealing insights into linear probe validity.
1 further paper
Oct 2026 · Universitat Autònoma de Barcelona, Apple
An AI model stores calculation results before writing individual digits
Uses causal subspace interventions to test whether visual and textual arithmetic share internal number concepts, challenging similarity-based probes.
6 of 10 papers shown