KV cache eviction retains a small token subset during inference to limit memory.
KV-cache eviction method is provably near-optimal, not just heuristic
DropKV decouples the NP-hard eviction search into per-token scores, yielding constant-factor guarantees and state-of-the-art speed and accuracy.
Academic
Aozhong Zhang · Selcuk Gurses · Yanxia Deng · Naigang Wang · Chi-Chun Liu · Davis Wertheimer · +5 more
University at Albany, SUNY · IBM T. J. Watson Research Center · Northwestern University
Research Digest··2 min read
Thread:KV Cache Compression
DropKV addresses the KV-cache memory bottleneck in long-context LLMs by turning the NP-hard problem of minimizing output perturbation from cache eviction into independent per-token scores.
Why this paper
From University at Albany, SUNY and 2 others · Part of KV Cache Compression, now 16 papers
In one line
DropKV makes KV-cache eviction tractable through independent per-token scoring while preserving near-optimal accuracy and enabling faster, memory-efficient long-context inference.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§