The authors built upon the beacon-based soft compression approach (Activation Beacon) and introduced low-rank constraints into the beacon KV projections.
Joint token-feature KV cache compression speeds long-context LLMs
SlimKV compresses both token count and per-token dimensions, enabling reconstruction-free attention via key-side RoPE removal for beacon states.
Chinese Tech
Zihan Teng · Jiayu Zhao · Wentao Ren · Minhao Fan · Tianrui Ma · Song Chen · +1 more
University of Science and Technology of China · Nanyang Technological University · Tencent
Research Digest··3 min read
Zihan Teng, Jiayu Zhao, Wentao Ren, Minhao Fan, Tianrui Ma, Song Chen, and Weichen Liu propose SlimKV, a KV-cache compression method that jointly reduces the number of cached states and their per-token dimensions.
Why this paper
From Tencent and 2 others
In one line
SlimKV jointly compresses tokens and features in KV cache, achieving high compression with reconstruction-free attention and speedups.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§