Nedelkoski et al.
E8 lattice vector quantization enables 2-bit keys for LLM KV-cache compression
Using structured 8D lattice codebooks, the method compresses keys to 2 bits while preserving attention fidelity and matching scalar decode throughput.
Academic
Sasho Nedelkoski · Alexander Acker · Soeren Becker · Dominik Scheinert · Odej Kao
Technical University of Berlin · logsight.ai
Research Digest··3 min read
Thread:KV Cache Compression
The authors apply E8 lattice vector quantization (VQ) to compress key vectors in LLM KV caches to 2 bits per dimension, achieving roughly 6x total compression when combined with 2-bit values.
Why this paper
From Technical University of Berlin and logsight.ai · Part of KV Cache Compression, now 13 papers
In one line
E8 lattice vector quantization with 2-bit keys and 2-bit values achieves 6x KV-cache compression while preserving near-baseline LLM quality.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§