5B.
Independent training runs tend to fix the same math problems
Across two language models, prior post-training outcomes predicted per-problem improvements better than pass rates and other pre-training signals.
Research Lab
Xiaoxian Duan
Institute of Automation, Chinese Academy of Sciences · Zhongguancun Academy · Z.ai
Research Digest··2 min read
5-billion-parameter base models on as many as 1,532 competition math problems.
Why this paper
From Institute of Automation, Chinese Academy of Sciences and 2 others
In one line
Independent post-training runs fix largely the same problems for a given base model, while common pre-training signals predict those gains poorly compared with existing checkpoints.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§