The authors analyzed the limitations of existing context distillation approaches.
Localized Context Distillation resolves the learning dilemma in interaction experience
The authors propose a method that identifies and revises only critical spans in a student's trajectory, outperforming both off-policy and on-policy context distillation across math, code, and agent tasks.
Chinese Tech
Zhezheng Hao · Yongkang Yang · Xilin Xia · Jiarui Yu · Yi Liu · Fanjunduo Wei · +6 more
Tencent · The Chinese University of Hong Kong · Zhejiang University · The Hong Kong University of Science and Technology
Research Digest··3 min read
Thread:Agent Self-Improvement
The authors introduce Localized Context Distillation (LoCD) to improve how large language models learn from interaction experience.
Why this paper
From Tencent and 3 others · Part of Agent Self-Improvement, now 56 papers
In one line
Localized Context Distillation improves LLM learning from interaction experience by having a teacher regenerate only critical spans of each student trajectory.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (4 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§