CORE tackles the two coupled problems in fixed-budget KV-cache compression: which states to keep and how to represent evicted information. It starts from an exact identity that decomposes eviction error into the amount of discarded attention mass and the directional gap between evicted and retained outputs, arguing that retention and compensation must be treated jointly, not separately.
Unified retention and compensation for KV cache eviction via coverage calibration
CORE combines offline set-level log-determinant coverage distillation with a lightweight indexer that drives both retention and evicted-mass redistribution, outperforming baselines by up to 3.78 points at 90% compression.
Hangzhou Institute for Advanced Study · University of Chinese Academy of Sciences
In one line
CORE unifies KV-cache retention and compensation by calibrating one allocation distribution that jointly selects Top-B states and writes latent memory from excluded mass.
Before you read this
Two ideas this paper builds onThe authors start from an exact identity: eviction error equals evicted attention mass times the directional gap between the evicted centroid and the retained output.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.