Research · Efficiency & systems
Language models can adapt compressed memory without growing their dictionaries
FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding
The authors combine faster selection of compression components with reuse of inactive components, keeping the dictionary fixed in size as generation continues.
MATH-500: 85.7% full-OMP support recall (as reported)
Rensselaer Polytechnic Institute, IBM Research