← All threads

Linear Probe Validity

Methods for determining when linear probes reliably reveal linear structure in hidden representations and how nonlinear features can masquerade as linear ones.

10 papers

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

How this thread developed

  1. Oct 2026 · KAIST, Helmholtz Munich

    Sparse coding replaces linear encoders to stabilize SAE concept discovery

    Replaces linear encorners in sparse autoencoders with per-ample sparse coding, stabbilizing concept discovery and challenging linear encorner assumptions.

  2. Oct 2026 · Cold Spring Harbor Laboratory, Bielefeld University

    Linear probes can mistake rich nonlinear features for linear representations

    Demonstrates that linear probes can mistake rich nonlinear features for linear representations, challenging a standard interpretability inference.

  3. Oct 2026 · Technion – Israel Institute of Technology

    Patch-token decoding does not reliably explain vision transformer decisions

    Tests whether patch token decoding reliably reveals class representations in vision transformers, finding it not guaranteed, thus questioning linear probe validity.

  4. Oct 2026 · Anthropic Fellows, Georgia Institute of Technology

    Many learned transcoder features closely reproduce individual transformer neurons

    Shows that many learned transcoder features closely replicate individual neurons, questioning claims that they discover novel structure beyond linear probes.

  5. 3 further papers

    Oct 2026

    Language models’ own confidence becomes harder to outperform as they improve

    Studies when a separate routing predictor can outperform the model's own confidence, revealing insights into linear probe validity.

  6. 1 further paper

    Oct 2026 · Universitat Autònoma de Barcelona, Apple

    An AI model stores calculation results before writing individual digits

    Uses causal subspace interventions to test whether visual and textual arithmetic share internal number concepts, challenging similarity-based probes.

6 of 10 papers shown