repeat shared blocks for deeper computation but incur linearly growing FLOPs and KV-cache memory. FlashLoop exploits lazy-update patterns: token state changes become sparse across loops, attention differences are dominated by few keys, and KV residuals are quantization-friendly. It applies token-sparse updates, , and KV-residual quantization to reduce computation and memory. Experiments show up to 1.64x end-to-end speedup and up to 6x KV-cache memory reduction while preserving accuracy, making looped transformers more practical for long contexts and many loops.
Training-free inference framework trims redundant computation and memory in looped Transformers
FlashLoop exploits lazy-update dynamics across loops to deliver up to 1.64x end-to-end speedup and 6x KV-cache reduction without retraining or accuracy loss
Research Lab
Wanqi Yang · Shiwei Liu
ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center
Research Digest··1 min read
Thread:Agent Harness Optimization
In one line
FlashLoop achieves up to 1.64x speedup and 6x KV-cache reduction in looped transformers without accuracy loss.
Before you read this
Three ideas this paper builds onlooped transformerA transformer architecture whose weights are reused across repeated passes, trading loop count for depth, compute, or accuracy.27 papers →kv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.82 papers →sparse attentionSelectively attending to only the most relevant positions or tokens to reduce computational overhead.51 papers →
Read it as
FlashLoop is a training-free inference framework that reduces cross-loop redundancy in looped Transformers through token-sparse updates, sparse attention, and KV-residual quantization.
The authors analyzed inference dynamics of , where a shared block is applied repeatedly to increase computational depth.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code