On-policy distillation (OPD) trains a student on teacher supervision for tokens on student-generated trajectories, but each trajectory follows only one continuation per decision.
Branching student trajectories improves on-policy distillation efficiency and accuracy
Chen et al. show that resampling continuations at informative decision points broadens teacher supervision and outperforms independent rollouts with fewer student tokens.
Chinese Tech
Dongyang Chen · Shudong Liu · Shubo Li · Duoxun Tang · Jinwei Liang · Zehuan Wu · +1 more
Tsinghua University · SenseTime Research · Peking University
Research Digest··2 min read
The authors propose BranchOPD, an on-policy distillation method that selects informative fork points using student uncertainty and teacher-student disagreement, then resamples student continuations from shared prefixes.
Why this paper
From SenseTime Research and 2 others
In one line
Branching student trajectories at uncertain decisions improves on-policy distillation accuracy on math and agent benchmarks while generating fewer tokens than independent rollouts.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§