The authors formalize off-policy and on-policy distillation as endpoints of a policy continuum.
Blending teacher and student token distributions improves distillation of language models.
The authors introduce Interpolated Policy Distillation, a controllable continuum between off-policy and on-policy rollout generation that outperforms both endpoints and heuristic segment interleaving.
Chinese Tech
Youxu Shi · Yifan Sun · Dacheng Yin · Haomiao Tang · Guangting Wang · Fengyun Rao · +2 more
Tencent Inc. · University of Science and Technology of China
Research Digest··3 min read
Shi et al.
Why this paper
From Tencent Inc. and University of Science and Technology of China · Part of Reasoning Distillation Alignment, now 7 papers
In one line
Interpolating student and teacher next-token distributions during rollout yields better distillation than using either endpoint alone.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§