ContextPilot addresses limitations in context management for long-horizon agentic tasks. Existing methods have a limited toolset (only search, deletion, summarization), inefficient exploration that treats all context edits equally, and coarse that rewards all intermediate edits uniformly. ContextPilot expands the toolset with planning, , and soft context offloading tools. It introduces an RL method that uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories passing through that edit. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming baselines across various base models and benchmarks. This is interesting because it enables LLM agents to manage their working context more intelligently, improving efficiency and effectiveness in complex, multi-turn tasks.
Agents learn to manage long contexts through fine-grained reinforcement learning
ContextPilot trains language-model agents to plan, store, compress, and remove context while assigning credit to individual editing decisions.
Chinese Tech
Zhuoshi Pan · Qizhi Pei · Junru Lu · Honglin Lin · H. Vicky Zhao · Di Yin · +1 more
Tsinghua University · Tencent Youtu Lab · Shanghai AI Lab
Research Digest··2 min read
In one line
ContextPilot uses fine-grained RL and an expanded toolset to proactively manage context, outperforming existing methods.
Before you read this
Three ideas this paper builds oncontext compactionCompressing or summarizing an LLM's past context into a smaller representation to extend memory and improve reasoning within limited token budgets.71 papers →agentic reinforcement learningTraining approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.102 papers →credit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.171 papers →
Read it as
Pan et al. introduce ContextPilot, a framework for controlling the growing working context of agents performing long-horizon tasks.
The authors expanded the context-management actions available to an agent beyond search, deletion, and summarization.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.