, TRM for Sudoku), MLP mixers, convnets, implicit models, and a looped language model (Huginn).
Compressed looped models settle, not drift, so precise final loops recover them
Kolawole et al. show that rounding error in looped models shifts the fixed point rather than accumulating, enabling predictable failure and recovery with few high-precision steps.
Top University
Steven Kolawole · Pearse Jim · Opegbemi M. Busoye · Glory Bagai · Virginia Smith
Carnegie Mellon University · ML Collective
Research Digest··3 min read
Thread:Precision Scaling Laws
Kolawole et al.
Why this paper
From Carnegie Mellon University and ML Collective · Part of Precision Scaling Laws, now 4 papers
In one line
Quantization errors in settling looped models shift their fixed point rather than accumulate, allowing collapse prediction and recovery with a few higher-precision loops.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (4 noted)
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§