Credit Assignment in Agentic RL
Methods for distributing credit over long trajectories in language-model agent reinforcement learning when only sparse terminal rewards are available.
30 papers · 3 months
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
17 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
June 2026 · IBM Research
RL with live server state boosts multi-step tool use in LLMs
Presents an RL framework for multi-step tool use that addresses credit assignment with live server state.
3 further papers
Aug 2026 · Beijing Jiaotong University, Peking University
Local milestones improve credit assignment for long-horizon language-model agents
Proposes MileGPO, using local milestones to provide intermediate credits for long-horizon tasks.
Aug 2026 · Fudan University, Tencent
Search agents improve longer when their critics evolve alongside them
Introduces CAFE, which alternates search and critic feedback, enabling credit assignment via online and offline RL.
3 further papers
Sept 2026 · Carnegie Mellon University, IBM Research
Dynamic rubrics improve credit assignment for long-horizon agent training
DRACO uses dynamic rubrics and stepwise rewards to improve credit assignment for long-horizon agent training when success is not programmatically checkable.
released code
Sept 2026 · Rochester Institute of Technology, Adobe Research
GRPO Can Reward Lucky Guesses as If They Were Reasoning
Shows GRPO can reward lucky guesses; proposes SignBalance to fix credit assignment when rewards are sparse.
2 further papers
Sept 2026 · Zhejiang University, Alibaba Group
Self-retiring distillation improves reinforcement learning for multi-turn agents
Trains a privileged teacher with environment rewards and distills to a student via on-policy distillation, improving credit assignment for multi-turn agents.
released code
Sept 2026 · University of Maryland, AWS AI Labs
Predicting environment observations during fine-tuning improves later agent exploration
ActObs predicts environment observations during fine-tuning, improving later exploration—a form of auxiliary supervision that supports credit assignment over agent trajectories.
2 further papers
Sept 2026 · Virginia Tech, University of Wisconsin Madison
Structural guidance improves language models’ long-horizon reasoning under sparse rewards
Proposes structural priors to guide exploration under sparse terminal rewards, outperforming baselines.
released code
Sept 2026 · University of Electronic Science and Technology of China, Independent Researcher
Anchored hypergraphs provide stable credit assignment for multi-agent teams
Introduces anchored overlapping sparse hypergraphs for temporally consistent value decomposition in multi-agent RL.
Sept 2026 · Beihang University, Zhongguancun Academy
Graph-based hindsight improves credit assignment for long-horizon language agents
Traces sparse rewards back to actions via graph-based hindsight, improving credit assignment in long-horizon settings.
2 further papers
Sept 2026 · Ant Group, Alibaba International Digital Commerce Group
Hierarchical supervision allocation improves long-horizon model distillation
Introduces hierarchical supervision allocation for long-horizon distillation, addressing credit assignment in student agent training.
5 further papers
Oct 2026 · Nanjing University, ByteDance
Historical trajectories provide step-level credit for training LLM agents via regressor
Uses historical trajectories to provide step-level credit for RL, directly advancing credit assignment methods.
12 of 29 papers shown