The authors identified two problems with standard on-policy distillation (OPD): token-wise supervision does not provide coherent alternative steps, and teacher feedback on degenerate prefixes can reinforce poor reasoning.
Segment-wise distillation improves LLM reasoning by learning to revise intermediate steps.
Replacing student reasoning segments with teacher redrafts boosts accuracy by 5.22% over baselines.
Chinese Tech
Yuxiang Zhang · Ding Cao · Shuting Cui · Lei Wang · Weijieying Ren · Tianxiang Zhao
The Hong Kong University of Science and Technology (Guangzhou) · University of Science and Technology of China · Alibaba Group · Stanford University
Research Digest··2 min read
The authors propose Seg-OPD, a method that trains large language models to revise their own intermediate reasoning steps by preferring teacher-written redrafts.
Why this paper
From Alibaba Group and 3 others
In one line
Segment-wise On-Policy Distillation improves reasoning by training students to prefer whole teacher redrafts of their reasoning segments.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§