The authors add four families of hidden pathways to four pretrained language models.
Causal interventions often succeed via hidden pathways, not natural mechanisms
A planted-pathway benchmark across four language models shows distance-based detectors score below chance, while a new causal test catches most wrong-mechanism successes.
Industry
Beiming Liu · Minjie Chen
China Financial Certification Authority · PetroChina Southwest Oil and Gasfield Company
Research Digest··3 min read
5-7B, then run 100,800 causal interventions to determine which succeed through these wrong mechanisms.
Why this paper
From China Financial Certification Authority and PetroChina Southwest Oil and Gasfield Company
In one line
Causal interventions routinely give right answers through wrong mechanisms, which existing site-level distance measures cannot detect.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§