The authors analyze how prior work evaluates KV cache reuse against full prefill, where the model recomputes attention states for the complete prompt.
Standard benchmarks overstate the accuracy of reused language model caches
The authors propose a context-sensitive evaluation method and a synthetic benchmark for testing difficult cache-reuse patterns.
Chinese Tech
Huawei Technologies Ltd. · ETH Zurich
Research Digest··3 min read
Cestola et al.
Why this paper
From Huawei Technologies Ltd. and ETH Zurich
In one line
Current evaluations of KV cache reuse overstate accuracy due to flawed metrics and datasets; a new methodology and benchmark reveal true costs.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§