Reinforcement learning for search agents rewards only the final answer, making it hard to assign credit to individual decisions.
Twin-world counterfactuals isolate decision effects in agentic reinforcement learning
Sharing sampling noise between factual and alternative rollouts yields unbiased, lower-variance credit for each decision.
Top University
Zeropy Surio · Tobias Engel · Mei-Lin Chou · Anika Raman · Samuel Okafor
University of Sheffield · Carnegie Mellon University · University of Michigan
Research Digest··3 min read
The authors present TWICE, a method for credit assignment in agentic RL that generates counterfactual rollouts sharing the same Gumbel noise as the factual episode.
§