struggle to select tools from vast real-world repositories because of context-length limits and functional similarities among tools. Existing reinforcement learning methods designed for knowledge-based question answering fail to model the multi-turn search and tool composition required. ToolSearcher introduces a novel RL framework with three components: category-constrained tool discrimination to better distinguish similar tools, event-level search modeling to explicitly optimize the discovery of target tools during search, and trajectory-aligned credit allocation to provide fine-grained rewards across different stages of the process. Experiments on large-scale benchmarks show consistent improvements over strong baselines in iterative search and complex tool composition settings.
Reinforcement learning improves tool selection across large repositories
ToolSearcher trains language models to search iteratively, distinguish similar tools and allocate credit across the full selection process.
Chinese Tech
Zhejiang University · Ant Group
Research Digest··2 min read
In one line
ToolSearcher uses reinforcement learning to optimize multi-turn tool selection at scale, outperforming existing methods.
Before you read this
Three ideas this paper builds onagentic reinforcement learningTraining approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.124 papers →multi-turn tool useThe capability of an agent to invoke external tools sequentially over multiple conversational turns to complete complex tasks.52 papers →credit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.190 papers →
Read it as
Dai et al. introduce ToolSearcher, a reinforcement learning framework for selecting tools when repositories are too large to place every tool description in the model's context.
The authors model tool selection as a multi-turn search problem rather than asking a language model to choose from a small, predefined list.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code