← All threads

KV Cache Compression

Methods for compressing the key-value cache in transformer inference to reduce memory while preserving reasoning quality.

26 papers

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

Results across this thread

32 reported results from the papers in this thread.

PaperBenchmarkMetricResultConditionsCode

The table is for subscribers. Subscribe to see every reported number side by side.

How this thread developed

  1. Oct 2026 · The Hong Kong University of Science and Technology (Guangzhou), Guangdong OPPO Mobile Telecommunications Corp., Ltd.

    Low-rank KV cache compression preserves all tokens during long reasoning

    Proposes iS-KV, an online low-rank KV-cache compression method that preserves recent tokens at full precision and folds older tokens into bounded-rank representations.

  2. Oct 2026 · Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences

    Unified retention and compensation for KV cache eviction via coverage calibration

    Derives an exact factorization of KV cache eviction error into evicted attention mass and directional gap, and proposes CORE for unified retention and compensation.

  3. 4 further papers

    Oct 2026 · University of Illinois Urbana-Champaign

    Low-precision caches preserve more reasoning history within fixed memory budgets

    Analyzes the trade-off between KV cache token count and bit-precision for long-reasoning models, finding that lower precision retains more reasoning history.

  4. Oct 2026 · Carnegie Mellon University, Capital One

    Selective KV-cache reuse depends on request history in rolling agent workloads

    Studies selective KV-cache reuse in rolling agent workloads, showing that request history affects outputs and that partial recomputation suffices.

  5. Oct 2026 · Amazon, University of California, Berkeley

    Query-adaptive cache reads preserve accuracy while moving fewer bits

    Proposes query-adaptive cache reads that transfer fewer bits while preserving accuracy.

  6. 1 further paper

    Oct 2026 · Huawei Technologies, University of Waterloo

    Kronecker product factorization compresses key-value caches up to 10x without accuracy loss.

    Caches keys and values as Kronecker product factors, compressing KV caches up to 10x without accuracy loss.

  7. 1 further paper

    Oct 2026 · Shanghai Jiao Tong University

    Recurrent memory signals predict which KV cache entries hybrid LLMs will need

    uses recurrent memory signals to predict future KV cache entries for selective caching

    released code

  8. 1 further paper

    Oct 2026 · Harbin Institute of Technology, Shenzhen Great Bay University

    Functional supervision improves cross-model KV cache transport over reconstruction alone

    Identifies that for zero-prefill cross-model context handoff, reconstruction fidelity is a poor proxy for downstream utility, offering a functional supervision alternative.

  9. 3 further papers

    Oct 2026 · Zuse Institute Berlin, Technische Universität Berlin

    KV-Surgeon optimizes LLM cache compaction by jointly selecting and compensating for deletions

    Adds KV-Surgeon, a compensation-aware compaction method that adjusts remaining cache entries after each deletion to preserve accuracy.

  10. 6 further papers

    Oct 2026 · Institute of Automation, Chinese Academy of Sciences, Tencent

    Language models can shrink fact-storage memory nearly eightfold, preserving most facts

    Introduces GateRQ, a compression method for large lookup tables in language models, reducing memory while preserving rare facts, relevant to cache compression.

10 of 26 papers shown