The authors studied sparse and continual RL fine-tuning of language models by treating attention heads as the fundamental trainable units.
Gradient norm ranks attention heads for efficient sparse and continual RL fine-tuning
Tuning a small subset of high-ranked heads can match dense baselines, while head partitioning reduces forgetting in sequential tasks.
Big Tech
Ruizhe Shi · Zaiwen Yang · Qiubo Wang · Banghua Zhu · Maryam Fazel
University of Washington · Tsinghua University · University of Hong Kong · Amazon, Inc.
Research Digest··3 min read
The authors propose using gradient norm to rank attention heads in language models, enabling sparse reinforcement learning fine-tuning that updates only a few heads.
Why this paper
From Amazon, Inc. and 3 others
In one line
Gradient norm ranks attention heads; tuning top heads matches dense fine-tuning, and partitioning heads prevents forgetting.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§