The authors distinguish two scales of agent behavior: generating individual tokens and taking complete actions, such as submitting a search query or calling a tool.
Training language agents across turns and tokens improves task performance
The authors report gains across search, tool-use and automation tasks by assigning credit across complete assistant turns while preserving token-specific learning signals.
Independent
Tao Ren · Hui Yang · Jia Deng · Tingyang Chen · Qiyao Wang · Shuo Tang · +13 more
Research Digest··2 min read
Ren et al. propose Multiscale Policy Optimization (MPO), a reinforcement-learning method for language-model agents that complete tasks through multiple interactions.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.