5 student models ranging from 7B to 72B parameters.
Smaller frozen models provide better rejected responses for preference distillation
Across students from 7B to 72B, using smaller Base models as reject sources outperforms self-generated rejects on code and math tasks with less inference compute.
Independent
Rui Cai · Wenhui Zhu · Xiwen Chen · Jincheng Cao · Han Yu · Shayan Mohajer Hamidi · +12 more
Research Digest··3 min read
The authors conducted a large-scale study on preference distillation, comparing reject sources from smaller frozen models versus the student's own responses.
Why this paper
Independent
In one line
Smaller frozen models generate better rejected responses for preference distillation than student-scale self-rejects, with less compute.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (params 14B)
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§