(OPD) transfers a large language model's capabilities to a smaller student using teacher guidance on the student's own rollouts. In long-horizon agentic tasks, uniform token-level matching wastes supervision on actions that are either irrelevant or beyond the student's current reach. This paper proposes LENS-OPD, a framework that allocates supervision hierarchically. It first adapts the rollout horizon to the student's skill (Locate), then tests whether teacher guidance at a specific decision actually improves the student's subsequent behavior (Validate), and finally concentrates token-level supervision on decisive teacher-student conflicts within the validated turn (Refine). Experiments on multiple agent benchmarks show that this approach consistently improves task performance over vanilla OPD and other baselines. The key insight is that effective supervision must sit at the intersection of future utility and current learnability, and this intersection evolves as the student learns.
Hierarchical supervision allocation improves long-horizon model distillation
The LENS-OPD method outperforms dense token-level supervision by dynamically selecting which student behaviors to correct based on future impact and learnability.
Chinese Tech
Yuhao Sun · Binrui Wu · Zhuoer Xu · Ming Wen · Haoxiang Xu · Bin Chen · +2 more
Ant Group · Alibaba International Digital Commerce Group · University of Science and Technology of China · Peking University · University of Electronic Science and Technology of China
Research Digest··3 min read
In one line
Hierarchical supervision allocation via Locate-Validate-Refine improves long-horizon on-policy distillation for LLM agents.
Before you read this
Two ideas this paper builds onon-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.131 papers →privileged informationInformation available only during training that improves performance but is not accessible at inference.85 papers →
Read it as
The authors propose LENS-OPD, a framework that distills a large teacher model into a smaller student agent for long-horizon tasks by hierarchically allocating supervision: first deciding how long the student should explore, then which decision to intervene on, and finally which tokens to correct.
Sun et al. formulate long-horizon (OPD) as a hierarchical supervision allocation problem.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.