The authors train looped language models by partitioning the backbone into input, shared recurrent, and output blocks.
A training recipe for looped language models that matches non-recurrent models at one loop and improves with more compute.
Elastic Latent Recurrence uses gated updates, within-step loop mixing, and loop conditioning to eliminate the one-loop tax, enabling a single checkpoint to trade inference compute for accuracy.
Big Tech
Faisal Ladhak · Shantanu Acharya · Santiago Akle Serano · Simeng Sun · Boris Ginsburg · Krishna C Puvvada
NVIDIA
Research Digest··3 min read
The authors present Elastic Latent Recurrence (ELR), a recipe for training looped language models that avoids the one-loop tax—the performance gap at a single loop compared to a non-recurrent model of the same size.
Why this paper
From NVIDIA
In one line
Elastic Latent Recurrence lets one language-model checkpoint match fixed-depth accuracy at one loop and improve accuracy by spending more inference compute on additional loops.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (params 4B)
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§