Linear Probe Validity
Methods for determining when linear probes reliably reveal linear structure in hidden representations and how nonlinear features can masquerade as linear ones.
4 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Oct 2026 · KAIST, Helmholtz Munich
Sparse coding replaces linear encoders to stabilize SAE concept discovery
Replaces linear encorners in sparse autoencoders with per-ample sparse coding, stabbilizing concept discovery and challenging linear encorner assumptions.
Oct 2026 · Cold Spring Harbor Laboratory, Bielefeld University
Linear probes can mistake rich nonlinear features for linear representations
Demonstrates that linear probes can mistake rich nonlinear features for linear representations, challenging a standard interpretability inference.
Oct 2026 · Technion – Israel Institute of Technology
Patch-token decoding does not reliably explain vision transformer decisions
Tests whether patch token decoding reliably reveals class representations in vision transformers, finding it not guaranteed, thus questioning linear probe validity.
Oct 2026 · Anthropic Fellows, Georgia Institute of Technology
Many learned transcoder features closely reproduce individual transformer neurons
Shows that many learned transcoder features closely replicate individual neurons, questioning claims that they discover novel structure beyond linear probes.