normally assigns one sparse outcome reward to an entire trajectory, which gives no signal about which intermediate tool calls or decisions were responsible for success. CBPO separates two problems that prior branch-based methods conflate: where to spend a fixed branch budget, and how to turn branch outcomes into token-level credit.
CBPO uses generation entropy across the full response to screen candidate branch positions, with path-level and node-level decay to keep exploration from collapsing onto a few paths or adjacent tokens. It then forms exact-prefix groups, where a parent trajectory and its branches share an identical token prefix, and defines Contrastive Branch Value (CBV) from reward variation within that group. CBV rescales continuation advantages without changing their sign, and selected nodes along the same trajectory partition it into non-overlapping credit segments to avoid duplicated gradients.
Requiring only outcome rewards and no process annotations, CBPO was evaluated on ten benchmarks, five mathematical reasoning and five knowledge-intensive search, at two model scales. It achieved the highest macro-average accuracy in both domains, outperforming state-of-the-art policy-optimization and branch-based baselines.