KV caches on NoPE-MLA models can be compressed before their queries exist by reading a salience signal hidden in the 64-dimensional decoupled branch. Existing methods that select tokens by observed attention fail there, scoring 0.00 and 0.33 needle retrieval at 8x compression, because the relevant queries have not happened yet.\n\nVestigeKV uses that branch as a query-independent ranking: it reads 11% of each row, partitions the cache into an attended tier and a GPU-resident archive, and admits the attended rows each step. On Kimi Linear 48B, retrieval stays at 1.00 at 8x and 0.96 at 32x from 8k to 65k context, matching full-row selection. The win is speed, not memory: the scan reads about 26% of dense attention's bytes, with a crossover near 40k context and up to 1.39x at 496k. The same operator under RoPE collapses to 0.08, so the result is specific to NoPE-MLA.
A Vestigial Attention Branch Signals Which Cache Entries to Keep
On Kimi Linear, a 64-dimensional branch identifies salient tokens before future queries arrive, enabling aggressive attended-cache compression without discarding archived entries.
In one line
On NoPE-MLA models, a 64-dimensional decoupled branch carries a query-independent salience signal that keeps needle retrieval at 1.00 under 8x and 0.96 under 32x compression.
Before you read this
Two ideas this paper builds onRead it as
Fan introduces VestigeKV, a training-free method for compressing the actively attended KV cache in a NoPE multi-head latent attention model.
VestigeKV ranks cache rows using Kimi Linear’s 64-dimensional decoupled attention branch, reading about 11% of each row.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.