The authors introduce ReAPO, which modifies GRPO by generating multiple attempts per rollout, where each subsequent attempt is conditioned on previous failures.
Retry-aware credit assignment boosts LLM reasoning efficiency and accuracy
The authors propose ReAPO, a modification to GRPO that generates retries after failures and assigns credit per attempt, improving first-attempt performance across benchmarks.
Top University
Nghi Dao · Calder Morton-Ferguson · Noah Golowich · Adam Block
Columbia University · Yale University · UT Austin
Research Digest··2 min read
The authors modify GRPO to incorporate failure-conditioned retries and assign credit using a stochastic shortest-path formulation.
§