The authors formalize token-level teacher selection in on-policy distillation, where a student generates a rollout and at each token a teacher is chosen to provide supervision.
Token-level teacher selection improves on-policy distillation for LLMs
A theoretical framework and new routing scheme yield up to 10 percentage point gains in pass@k across math, code, and reasoning tasks.
Big Tech
Stanley Wei · Jiin Woo · Kaan Ozkara · Youngsuk Park
Princeton University · Amazon Annapurna Labs
Research Digest··3 min read
The authors propose a framework for token-level teacher selection during on-policy distillation (OPD), where different teachers can supervise different tokens of a student rollout.
Why this paper
From Amazon Annapurna Labs and Princeton University · Part of Reasoning Distillation Alignment, now 31 papers
In one line
Token-level teacher selection in on-policy distillation improves student capability and diversity by over 10 percentage points on several tasks.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§