The authors formalize test-time scaling methods and introduce the concept of an attractor: a set of answers that, once reached, the sequential method is unlikely to leave.
Sequential test-time scaling can surpass parallel scaling by escaping attractors
A model-mixing intervention helps language models avoid premature convergence and find answers beyond those accessible through parallel sampling.
Top University
Joseph Rance · Fabio Pizzati · Juil Sock · Woody Bayliss · Marc Górriz Blanch · Philip Torr · +1 more
University of Oxford · MBZUAI · BBC R&D
Research Digest··2 min read
Thread:Test-Time Scaling Dynamics
Rance et al.
Why this paper
From University of Oxford and 2 others
In one line
Sequential test-time scaling prematurely converges to attractors, but model-mixing escapes them, outperforming parallel scaling.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§