Large language models (LLMs) can call external tools to fetch information or perform actions, but they often misuse tools, for instance, calling a tool without needed information or pretending to have data they don't. StateAlign trains an LLM to first assess its current situation, what it knows, what tools are available, and what the user has requested, before deciding whether to call a tool, ask the user, or decline. This improves the model's ability to complete tasks correctly, recover from mistakes, and avoid inventing tool outputs, as shown by higher accuracy on benchmarks and lower rates of tool hallucination.
AI agents can spot missing information yet still fabricate tool inputs
Lu and colleagues report that training agents to distinguish situations requiring different actions improves task completion while reducing unsupported tool use.
Big Tech
Zhexi Lu · Subhajit Chaudhury · Tejaswini Pedapati · Keerthiram Murugesan · Lei Yu
Rensselaer Polytechnic Institute · IBM Research
Research Digest··2 min read
In one line
StateAlign uses reinforcement learning to train multi-turn tool agents to align decisions with the current task state.
Before you read this
Three ideas this paper builds onreinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.289 papers →multi-turn tool useThe capability of an agent to invoke external tools sequentially over multiple conversational turns to complete complex tasks.49 papers →contrastive alignmentA method that learns embeddings by minimizing distance between positive pairs and maximizing distance between negative pairs.46 papers →
Read it as
An AI agent can correctly identify missing information in its reasoning, then call a tool using fabricated arguments anyway.
The authors developed StateAlign, a reinforcement-learning approach that rewards both successful task completion and decisions appropriate to the information and tools currently available.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.