The authors sampled reasoning rollouts from OLMo2 and Qwen3 models and identified n-grams associated with incorrect outcomes.
Language model RL failures can guide targeted pretraining repairs
Diagnostic phrases in failed reasoning traces retrieved a compact training corpus that improved subsequent reinforcement learning across model families.
Top University
Xinyi Wang · Danqi Chen
Princeton Language and Intelligence
Research Digest··3 min read
Wang and Chen show that reinforcement learning with verifiable rewards fails in recognizable, localized ways: a small set of phrases and operations strongly predicts whether a reasoning rollout will end incorrectly.
§