The authors introduce ATLAS, a method that uses an activation atlas built from retained-domain representations to condition a shared low-rank residual during fine-tuning. Each token is routed to nearby atlas charts, and the chart's reference center and local directions filter the update, making adaptation input-dependent. On Qwen3-8B, ATLAS achieves lower mean retained-output KL divergence than all seven published baselines at matched coding performance, with consistent advantages across training seeds. Answer-level analyses show fewer rewritten mathematical answers and more stable commonsense choices. The approach is practical: it requires compact storage and adds modest decoding overhead, enabling specialized skill acquisition while preserving existing behavior across multiple model families and retained domains.
Input-conditioned fine-tuning preserves language model behavior on unrelated tasks
The ATLAS method uses activation atlases to direct task-specific updates only where needed, reducing answer rewriting.
Chinese Tech
Jiangtao Lin · Bangyang Wei · Yihang Ding · Siyi Liu · Yuhan Dong
Tsinghua University · SZ DJI Technology Co., Ltd. · Tencent Holdings Limited
Research Digest··2 min read
Thread:Tail-Safety Fine-Tuning
In one line
ATLAS fine-tunes language models with less rewriting of answers outside the target task by conditioning updates on local retained geometry.
Before you read this
Two ideas this paper builds oncontinual learningA learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.72 papers →activation steeringActivation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.83 papers →
Read it as
Lin et al. introduce ATLAS, a method for fine-tuning language models that preserves behavior on retained domains while learning new tasks.
The authors propose ATLAS, which constructs an activation atlas from retained-domain representations.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.