Agents learn to manage long contexts through fine-grained reinforcement learning

ContextPilot trains language-model agents to plan, store, compress, and remove context while assigning credit to individual editing decisions.

PaperChinese Techcs.CLarXiv:2608.28476v1
Zhuoshi Pan · Qizhi Pei · Junru Lu · Honglin Lin · H. Vicky Zhao · Di Yin · +1 more

Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab

Research Digest··2 min read
Pan et al. introduce ContextPilot, a framework for controlling the growing working context of agents performing long-horizon tasks. On long-context question answering and deep-search benchmarks, the authors report better task performance than existing baselines while retaining a more compact context.

What they did

The authors expanded the context-management actions available to an agent beyond search, deletion, and summarization. ContextPilot adds global planning, long-term memory, and “soft offloading,” which compresses or moves information out of the immediate working context without simply discarding it.

They also developed a reinforcement-learning procedure that uses changes in context and output entropy to identify consequential editing decisions. It samples alternative branches at those points and estimates an action-level advantage from the branched trajectories passing through each context-editing action, rather than applying one final reward uniformly to every action.

Key findings

  • ContextPilot outperformed the evaluated context-management baselines on long-context question-answering and deep-search tasks.
  • The framework achieved stronger task performance while maintaining a smaller working context.
  • The reported gains were consistent across multiple base models and benchmarks, although the abstract does not provide model names, benchmark names, or numerical effect sizes.

Why it matters

Long-running agents can accumulate more history than is useful or affordable to process. This work suggests that context management can be learned as a sequence of distinct, outcome-sensitive decisions, potentially allowing agents to operate over longer tasks without preserving every prior interaction in their active prompt.

Caveats

The supplied abstract does not report absolute scores, context-length reductions, computational costs, or comparisons by individual tool, so the size and source of the gains cannot be assessed here. It also remains unclear how well the learned policy transfers beyond the evaluated question-answering and search settings, or whether branching during training introduces substantial additional expense.

§
newspaper

Research Digest

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.