KV Cache Compression
Methods for compressing the key-value cache in transformer inference to reduce memory while preserving reasoning quality.
26 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
32 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Oct 2026 · The Hong Kong University of Science and Technology (Guangzhou), Guangdong OPPO Mobile Telecommunications Corp., Ltd.
Low-rank KV cache compression preserves all tokens during long reasoning
Proposes iS-KV, an online low-rank KV-cache compression method that preserves recent tokens at full precision and folds older tokens into bounded-rank representations.
Oct 2026 · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences
Unified retention and compensation for KV cache eviction via coverage calibration
Derives an exact factorization of KV cache eviction error into evicted attention mass and directional gap, and proposes CORE for unified retention and compensation.
4 further papers
Oct 2026 · University of Illinois Urbana-Champaign
Low-precision caches preserve more reasoning history within fixed memory budgets
Analyzes the trade-off between KV cache token count and bit-precision for long-reasoning models, finding that lower precision retains more reasoning history.
Oct 2026 · Carnegie Mellon University, Capital One
Selective KV-cache reuse depends on request history in rolling agent workloads
Studies selective KV-cache reuse in rolling agent workloads, showing that request history affects outputs and that partial recomputation suffices.
Oct 2026 · Amazon, University of California, Berkeley
Query-adaptive cache reads preserve accuracy while moving fewer bits
Proposes query-adaptive cache reads that transfer fewer bits while preserving accuracy.
1 further paper
Oct 2026 · Huawei Technologies, University of Waterloo
Kronecker product factorization compresses key-value caches up to 10x without accuracy loss.
Caches keys and values as Kronecker product factors, compressing KV caches up to 10x without accuracy loss.
1 further paper
Oct 2026 · Shanghai Jiao Tong University
Recurrent memory signals predict which KV cache entries hybrid LLMs will need
uses recurrent memory signals to predict future KV cache entries for selective caching
released code
1 further paper
Oct 2026 · Harbin Institute of Technology, Shenzhen Great Bay University
Functional supervision improves cross-model KV cache transport over reconstruction alone
Identifies that for zero-prefill cross-model context handoff, reconstruction fidelity is a poor proxy for downstream utility, offering a functional supervision alternative.
3 further papers
Oct 2026 · Zuse Institute Berlin, Technische Universität Berlin
KV-Surgeon optimizes LLM cache compaction by jointly selecting and compensating for deletions
Adds KV-Surgeon, a compensation-aware compaction method that adjusts remaining cache entries after each deletion to preserve accuracy.
6 further papers
Oct 2026 · Institute of Automation, Chinese Academy of Sciences, Tencent
Language models can shrink fact-storage memory nearly eightfold, preserving most facts
Introduces GateRQ, a compression method for large lookup tables in language models, reducing memory while preserving rare facts, relevant to cache compression.
10 of 26 papers shown