Encoder loop adds compute without growing memory in language models

The technique, applied to a 552B-parameter model, cuts per-token memory by 36.8× and fits 38 requests where none could fit before.

Independent

Jiaqi Leng · Minshen Zhang · Zhaofeng Sun · Yu-Yang Qian · Peng Zhao · Hao Zhang

Research Digest·12 November 2026·2 min read

Thread:Compute-Efficient Scaling

Illustration: Encoder loop adds compute without growing memory in language models

In one line

Looped CEDs achieve recurrent depth with zero round-wise KV growth and match looped Transformer quality.

Before you read this

Three ideas this paper builds on

looped transformerA transformer architecture whose weights are reused across repeated passes, trading loop count for depth, compute, or accuracy.27 papers →kv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.79 papers →test-time scalingAllocating extra computation during inference to improve output quality.114 papers →

Read it as

Large language models store a memory of the conversation so far, called a key-value cache. This cache grows with each processing step, filling memory and limiting how many users can be served.

The authors built a 'Looped CED' model that adds extra reasoning passes without increasing the cache. They added loops in the part that builds the global memory and trained them to converge, so one cache works for all passes.

This model matched deeper models in quality while using 36.8 times less memory per word. A version with 552 billion parameters could serve 38 long requests at once, boosting throughput by 3.6 times.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

looped transformer kv cache compression causal encoder-decoder fixed-point convergence long-context processing test-time scaling