Standard on-policy distillation trains a student on its own generated responses while using a teacher to provide guidance at every token.
Reward-guided token weighting makes language-model distillation more effective
The authors show that weighting teacher feedback by its predicted effect on task reward improves mathematical reasoning and code generation across several student models.
Big Tech
Yilan Chen · Zian Su · Manish Prajapat · Dingmin Wang · Rein Houthooft · Rui Song · +1 more
UC San Diego · Purdue University · AWS AI Labs · Amazon
Research Digest··2 min read
Chen et al.
Why this paper
From AWS AI Labs and 3 others
In one line
Reward-Aligned Distillation weights teacher tokens by predicted effect on student reward and improves math and code reasoning at matched training budgets.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§