The authors evaluated LoopCD across four families of looped Transformers.
Earlier Recurrent States Improve Looped Transformer Decoding Without Retraining
LoopCD contrasts early and final recurrent predictions to improve reasoning and coding accuracy, while allowing fewer loops at inference.
Big Tech
Weihao Liu · Huangjie Zheng · Tianrong Chen · Rohit Dilip · Richard He Bai · Yizhu Jiao · +2 more
Apple
Research Digest··2 min read
Liu et al.
Why this paper
From Apple
In one line
LoopCD contrasts early and final loop predictions to improve accuracy and allow halving recurrent loops for reduced compute.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§