The authors formalized a strict “linear world model” as a representation whose dominant variance is an affine function of known latent factors.
Linear probes can mistake rich nonlinear features for linear representations
Three diagnostics show that accurate decoding does not establish that a factor dominates a model’s representational geometry.
Academic
Nadir Kutluozen · Christian Internò · David Klindt
Cold Spring Harbor Laboratory · Bielefeld University
Research Digest··3 min read
Kutluozen, Internò and Klindt test a common inference in representation learning: that a factor decoded accurately by a linear probe must form a major axis of the representation.
§