1 families on long reasoning trajectories while varying three factors: whether trajectories were correct, how many distinct prompts generated them, and whether the student could already solve the training problems.
Long reasoning distillation may teach behavior more than factual knowledge
Experiments across three model families suggest that structured reasoning traces remain useful despite factual errors or limited prompt diversity.
Research Lab
Yongcan Yu · Jian Liang · Lingxiao He · Kuangpu Guo · Yanbo Wang · Shuo Lu · +2 more
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan · University of Science and Technology of China
Research Digest··2 min read
Yu and colleagues test whether long chain-of-thought distillation depends on correct answers and broad problem coverage, two assumptions behind common rejection-sampling pipelines.
Why this paper
From Institute of Automation, Chinese Academy of Sciences and 3 others
In one line
Long-CoT distillation works by teaching models structured reasoning behavior, not by memorizing correct facts or broad problem coverage.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§