8-27B and defined future KV utility by how much removing an entry changes attention output on the model's own continuations.
Recurrent memory signals predict which KV cache entries hybrid LLMs will need
GateSight uses Gated DeltaNet write and decay signals to evict attention cache entries with no query, keeping a tenth of the cache.
Top University
Junjie Li · Jiong Lou · Jie Li
Shanghai Jiao Tong University
Research Digest··2 min read
Thread:KV Cache Compression
The authors show that Gated DeltaNet write and decay signals in hybrid LLMs predict which attention KV entries will be needed later, before any query is seen.
Why this paper
From Shanghai Jiao Tong University · Released code · Part of KV Cache Compression, now 13 papers
In one line
GateSight uses GDN recurrent memory signals to predict attention KV utility, enabling 90% cache reduction with top accuracy.
What it released
Code
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§