The paper studies how the diversity of reasoning steps in supervised fine-tuning (SFT) data affects a model's ability to improve through reinforcement learning (RL). It proposes a lightweight, rule-based fingerprint to measure route diversity and selects SFT traces that are diverse rather than similar.
On synthetic experiments with OLMo3-7B, selecting diverse SFT traces improved pass@8 by 16.9 points on environments held out from SFT. In a single-model condition, diverse selection gained up to 6.2 points of mean pass@8 across 10 mathematics benchmarks. The approach outperformed more expensive alternatives on three open-source corpora, all while using only CPU resources.
This is interesting because it identifies a practical, cheap criterion—reasoning-route diversity—that better prepares models for RL than existing selection methods. Pre-RL diagnostics suggest diverse SFT provides more prompts with both successful and failed attempts, giving RL a richer learning signal.