Multi-head latent attention stores one compact latent representation per token, then projects it into keys and values.
Joint calibration makes 2-bit latent attention caches more accurate and faster
RoLaQ jointly tunes latent rotations and quantization ranges, improving accuracy across three models while increasing long-context serving capacity and throughput.
Chinese Tech
Shiyu Ji · Peng Ling · Chen Lin · Shupeng Li · Yixuan Wang · Qingfu Zhu · +1 more
Harbin Institute of Technology · Tsinghua University · Baidu Inc.
Research Digest··3 min read
Ji et al.
Why this paper
From Baidu Inc. and 2 others
In one line
RoLaQ jointly calibrates rotation and quantization ranges for 2-bit MLA KV caches, improving accuracy across three MLA models and boosting throughput.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (hardware A100 80GB)
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§