The authors address thinking collapse in on-policy self-distillation (OPSD), where a teacher conditioned on privileged information produces accurate but shortcut trajectories unsuitable for student learning.
Shaping teacher behavior improves self-distillation for language models
The TeachBack framework uses bidirectional preference optimization to encourage teachers to produce exploration-rich reasoning trajectories that students can learn from.
Chinese Tech
Linfei Ji (Huazhong University of Science and Technology) · Jiayun Li (Tsinghua University) · Haozhao Wang (Huazhong University of Science and Technology) · Siyu Jiang (Huazhong University of Science and Technology) · Zehao Wang (Harbin Institute of Technology) · Longtao Xiao (Huazhong University of Science and Technology) · +2 more
Research Digest··2 min read
The authors propose TeachBack, a method for on-policy self-distillation that shapes the teacher's generation behavior by controlling the length of reference reasoning and prompt style.
§