looped transformer
A transformer is a neural network that reads sequences in layers. A looped transformer reuses the same block of layers several times, feeding each pass's output back in as the next input. That adds computation and depth without new parameters; loop count can be chosen per request. The danger is unstable states, so what gets carried around the loop matters.
The papers find that naive repetition is not enough. Learned ways to mix old and new states, compressing or sparsifying the carried state, and using an early loop's prediction as a contrast make later loops genuinely useful. One checkpoint then serves many depths: deeper loops improve accuracy, memory stays bounded, and users trade latency for quality.
- Papers
- 27
- Released code
- 2
- First seen
- Oct 2026
- Latest
- Nov 2026
27 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Halting heads can overstate compute savings in looped language models
Halting heads in looped language models trained with mixture loss save almost no compute under their own exit rule.
Oct 2026 - 2Reusing layers in trained AI models pays off only for the smallest models
A loop added after training pays for its extra inference compute only for the smallest models; larger models gain nothing over equal-parameter or equal-compute baselines.
Nov 2026 - 3Encoder loop adds compute without growing memory in language models
Looped CEDs achieve recurrent depth with zero round-wise KV growth and match looped Transformer quality.
Nov 2026
How fast it is moving
Often studied with
test-time scalingAllocating extra computation during inference to improve output quality.12 togethermechanistic interpretabilityThe study of how neural networks compute through internal circuits, features, and causal interventions, rather than just input-output behavior.3 togetherkv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.3 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Independentcs.LG
Encoder loop adds compute without growing memory in language models
Looped CEDs achieve recurrent depth with zero round-wise KV growth and match looped Transformer quality.
Nov 2026
- Big Techcs.LG
Reusing layers in trained AI models pays off only for the smallest models
A loop added after training pays for its extra inference compute only for the smallest models; larger models gain nothing over equal-parameter or equal-compute baselines.
Microsoft Applied Sciences Group · Nov 2026
- Big Techcs.CL
Halting heads can overstate compute savings in looped language models
Halting heads in looped language models trained with mixture loss save almost no compute under their own exit rule.
Apple · Oct 2026
- Top Universitycs.LG
Language models benefit from repeated layers, but more can hurt
Looping transformer layers improves parameter efficiency but requires careful design to avoid quality loss and parallel inference slowdown.
FAIR at Meta, MIT · Oct 2026
- Big Techcs.CL
Reusing computations across model depth favors learning from the prompt
Depth-grown and looped language models rely more on in-context learning, acquire it earlier, and retain it more strongly than standard dense models.
Technical University of Munich, Helmholtz AI · Oct 2026
- Big Techcs.CV
AI models can reuse layers while retaining image and text abilities
CLIP models tolerate substantial reuse of transformer blocks across depth while preserving transferable capabilities.
Samsung AI Center Warsaw, AGH University of Krakow · Oct 2026
- AI Startupcs.LG
Shared KV-caches cut looped-model inference memory without quality loss
Final-recurrence KV-caches can be reused across recurrences in looped models without quality loss when training induces contractive residual dynamics and corrects first-recurrence errors.
University of California, San Diego, Together AI · Oct 2026
- Big Techcs.AIcode
Spherical geometry simplifies adaptive-loop reasoning and improves halting efficiency
Spherical Fixed-Point Reasoners improve adaptive loop halting by running damped fixed-point iterations on the unit hypersphere.
EPFL, Lamarr Institute · Oct 2026
- Big Techcs.CL
A training recipe for looped language models that matches non-recurrent models at one loop and improves with more compute.
Elastic Latent Recurrence lets one language-model checkpoint match fixed-depth accuracy at one loop and improve accuracy by spending more inference compute on additional loops.
NVIDIA · Oct 2026
- Research Labcs.LG
Slow oscillatory modes bottleneck recurrent reasoning; a rotational symmetry sidesteps them
A slowly decaying oscillatory mode can bottleneck convergence in looped reasoners; QRR makes symmetry-generated motion irrelevant to quotient dynamics to avoid it.
ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems · Oct 2026
- Top Universitycs.AI
Looped decision models spend extra computation on harder questions
LEV lets a decision model improve accuracy on hard questions by looping its layers, with a stopper saving computation per question.
Lehigh University, University of British Columbia · Oct 2026
- Chinese Techcs.LG
Asymmetric compute allocation improves looped Mixture-of-Experts models
Compressing the first loop visit and spending the saved compute on hidden width lowers validation loss in looped mixture-of-experts models across budget sizes.
SJTU, Shanghai AI Lab · Oct 2026
- Chinese Techcs.LG
Upcycling pretrained LLMs with layer-wise loops breaks the two-loop saturation limit
UpLoop converts a pretrained non-looped LLM into a recurrent model, raising 20-benchmark average accuracy from 51.90 to 58.07 with three local executions.
Inclusion AI, Ant Group, Shanghai Jiao Tong University · Oct 2026
- Independentcs.AI
Loop-native residuals let Transformers reason across tens of thousands of iterations
Looped Transformers keep improving past 20,000 test-time iterations when residual connections learn to retain and weight past recurrent states.
Oct 2026
- Big Techcs.LG
Earlier Recurrent States Improve Looped Transformer Decoding Without Retraining
LoopCD contrasts early and final loop predictions to improve accuracy and allow halving recurrent loops for reduced compute.
Apple · Oct 2026
- Top Universitycs.AI
Recursive latent reasoner boosts diffusion models for visual puzzles
A recursive reasoner coupled to a frozen diffusion model solves visual reasoning puzzles from pixels without symbolic supervision.
Warsaw University of Technology, IDEAS Research Institute · Oct 2026
- Top Universitycs.LG
Compressed looped models settle, not drift, so precise final loops recover them
Quantization errors in settling looped models shift their fixed point rather than accumulate, allowing collapse prediction and recovery with a few higher-precision loops.
Carnegie Mellon University, ML Collective · Oct 2026
- Academiccs.AI
A tiny LoRA intervention unlocks deep reference-following in transformers
A tiny rank-8 LoRA at one early layer lets pretrained transformers follow 24-line reference chains with 99% accuracy instead of 15.5%.
Georgia Institute of Technology · Oct 2026
- Big Techcs.AI
Reusing Ranking Blocks Cuts Communication While Preserving Short-Video Quality
CDS reduces parameters by 26.6% and improves training throughput by 6.4% in a production ranking system.
Google · Oct 2026
- Chinese Techcs.LG
A model can become its own stronger teacher by looping selected layers
LoOPSD uses repeated layers as a teacher to improve student model reasoning through on-policy self-distillation.
Anhui Province Key Laboratory of Digital Security, Token Foundry, Alibaba Group · Oct 2026
- Chinese Techcs.CV
Reusing Transformer blocks improves image generation without enlarging the model
Looped diffusion transformers with deep supervision and self-modulating attention outperform much larger models with less compute.
SenseTime Research, LeapLab, Tsinghua University · Oct 2026
- Big Techcs.LG
Shared memory improves looped Transformers while shrinking context caches
Shared memory across recursions improves quality and reduces memory in looped transformers.
IBM Research, Cornell University · Oct 2026
- Top Universitycs.LG
Loop-specific targets let diffusion transformer improve with extra inference loops
LiFT improves ImageNet generation by repeatedly applying a shared DiT core with depth-specific trajectory targets, cutting parameters and compute versus a dense DiT baseline.
University of Amsterdam, TNO · Oct 2026
- Big Techcs.LG
Looping and sparse experts provide complementary routes to efficient scaling
Looped MoE scaling follows a bounded, sparsity-conditional recurrence law that subsumes dense and MoE scaling laws and predicts held-out loss.
Meta AI · Oct 2026
- Research Labcs.LGcode
Training-free inference framework trims redundant computation and memory in looped Transformers
FlashLoop achieves up to 1.64x speedup and 6x KV-cache reduction in looped transformers without accuracy loss.
ELLIS Institute Tübingen, Max Planck Institute for Intelligent Systems · Oct 2026
- Big Techcs.AI
Adaptive token computation narrows diffusion language models’ quality gap
Diffusion language models can match autoregressive quality by allocating more recurrent compute to hard tokens while easy tokens commit early.
University of Illinois Chicago, Amazon AGI · Oct 2026
- Chinese Techcs.CV
Three-stage latent generator reaches 1.11 FID on ImageNet
A distill-then-refine latent rollout approach with a staged Transformer achieves FID 1.11 on ImageNet 256x256 without Fréchet losses.
ByteDance · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.