with verifiable rewards (RLVR) improves single-sample accuracy for large reasoning models but often leaves pass@k unchanged, which limits how much test-time search can help. This work analyzes the structure of train-time rollouts and derives three principles: rollout budget should depend on problem difficulty because more rollouts help on hard problems but hurt on easy ones; tree-structured rollouts discover correct answers more efficiently than independent parallel sampling; and branching at sentence-level entropy produces more semantically diverse reasoning paths than token-level entropy branching.
Based on these principles, DATPO combines difficulty-adaptive tree search with an annealed sibling-diversity term in the advantage estimate, explicitly rewarding branches that are semantically different. On mathematical reasoning benchmarks DATPO outperforms the baselines especially in pass@k, and this coverage gain carries over to test-time scaling performance. The result is interesting because it moves the lever for reasoning coverage from the optimizer or reward to the structure of rollouts themselves.