TASPO is a method for training language-model agents. Standard outcome-based RL such as GRPO gives the same terminal advantage to every action in a trajectory, even when some actions helped and others did not. TASPO uses from successful sibling trajectories to refine this assignment. It filters that information for applicability to the current interaction, aggregates likelihood shifts per executable action, and uses the resulting relative support to reweight the original advantage. Because the weights are positive, bounded, and mean-one, the verified outcome still controls the update direction and average scale; privileged information only redistributes credit across actions. On ALFWorld, Search-QA, and WebShop, across three backbone models, TASPO improves over GRPO by an average of 10.6% under matched interaction budgets and update counts. The analyses also show action-level assignment stabilizes training and reduces supervision mismatch.
Privileged supervision improves action-level credit for language-model agents
TASPO redistributes verified trajectory rewards across executable actions using training-only information while preserving the outcome-based update direction and average scale.
Top University
Jingxiao Yang · Wangjie Gan · Yingxuan Zhuang · Wenqi Zhang · Jintao Chen · Xuhong Zhang
Zhejiang University
Research Digest··2 min read
In one line
TASPO converts privileged process supervision into outcome-grounded action credit, improving agentic policy optimization by 10.6% over GRPO across three benchmarks.
Before you read this
Three ideas this paper builds oncredit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.171 papers →agentic reinforcement learningTraining approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.102 papers →group relative policy optimizationA reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.154 papers →
Read it as
Yang et al. address a credit-assignment problem in reinforcement learning for agents: a verified task outcome is typically applied uniformly to every decision in a long interaction.
The authors developed TASPO, a policy-optimization method that combines verified outcome rewards with process-level supervision available only during training.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.