The authors propose ATLAS, which constructs an activation atlas from retained-domain representations.
Input-conditioned fine-tuning preserves language model behavior on unrelated tasks
The ATLAS method uses activation atlases to direct task-specific updates only where needed, reducing answer rewriting.
Chinese Tech
Jiangtao Lin · Bangyang Wei · Yihang Ding · Siyi Liu · Yuhan Dong
Tsinghua University · SZ DJI Technology Co., Ltd. · Tencent Holdings Limited
Research Digest··2 min read
Lin et al.
Why this paper
From Tencent Holdings Limited and 2 others
In one line
ATLAS fine-tunes language models with less rewriting of answers outside the target task by conditioning updates on local retained geometry.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§