The authors first built a synthetic framework with known ground-truth dictionaries and coefficients to test when sparse autoencoders (SAEs) recover true features.
Sparse coding replaces linear encoders to stabilize SAE concept discovery
The authors show that optimizing coefficients per sample before each dictionary update yields more consistent dictionaries across random seeds in vision and language models.
Top University
Jinho Choi · Hyesu Lim · Jaegul Choo · Steffen Schneider
KAIST · Helmholtz Munich
Research Digest··3 min read
Choi, Lim, Choo, and Schneider propose Consistent Sparse Dictionary Learning (ConSDL), which swaps the linear encoder of sparse autoencoders for per-sample sparse coding.
§