The authors formalize intervention attribution as the expected dot product between the intervention edit and the gradient of the model output.
Observational discovery of sparse autoencoder features does not predict their causal effects
The authors formalize intervention attribution and show that standard correlation-based scores systematically misestimate the impact of steering or ablation, while gradient-based attribution closely tracks realized interventions.
Top University
Yifeng Guo · Yanran Li · Xiang Chen
St. Jude Children's Research Hospital · Columbia University
Research Digest··2 min read
The authors define intervention attribution as the first-order effect of a specified intervention on model output, prove that observational feature-discovery scores (correlations, probe coefficients) generally do not identify it, and demonstrate across four model-SAE pairs that gradient-based attribution predicts real intervention outcomes with high accuracy while probe-implied changes are off by orders of magnitude.
Why this paper
From Columbia University and St. Jude Children's Research Hospital
In one line
Observational feature discovery does not identify intervention effects; gradient-estimated attributions better predict ablation and steering outcomes.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§