The paper studies for multi-turn agents trained with terminal verifiable rewards. It compares uniform dense reward, binary outcome reward, and concentrated reward (targeted on progress or random turns) across multiple benchmarks and model families. The key finding is that uniform dense reward consistently beats binary reward, and that concentrating reward onto specific turns—even those deemed 'progress'—is equally harmful as random concentration. The binary outcome reward is often net-harmful, degrading performance.
The mechanism is coverage: terminal-state verifiers only observe the final write turn (k=1 in 98% of rollouts), but tasks require a 5–8 step causal chain. Concentrating reward on a few turns starves the rest; uniform spreading covers the full chain. This challenges the prevailing approach of adding per-turn reward models or critics, suggesting that simple uniform redistribution is a strong default baseline.