Speech language models can drift as audio retrieval mistakes accumulate

Across four speech language models, the authors report that speech-aware retrieval improves long-recording accuracy under limited access to stored audio representations.

Big Tech

Dancheng Liu · Markus Müller · Rupak Vignesh Swaminathan · Jing Liu · Srikanth Ronanki · Aram Galstyan

University at Buffalo · Amazon AGI

Research Digest·3 November 2026·2 min read

Illustration: Speech language models can drift as audio retrieval mistakes accumulate

In one line

C3 outperforms other KV retrieval methods on long-form speech tasks by clustering audio keys and having heads compete.

Before you read this

Two ideas this paper builds on

kv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.55 papers →sparse attentionSelectively attending to only the most relevant positions or tokens to reduce computational overhead.39 papers →

Read it as

Speech language models process audio much faster than text, so they store a large memory of past sounds. This memory, called a key-value cache, grows huge with long audio and slows down generation while using a lot of memory.

The researchers developed C3, which organizes the cache into clusters and lets parts of the model compete for access to the most useful sections. They tested it on speech recognition, translation, and summarization using several different speech language models.

Under tight memory limits, C3 beat all other cache-reduction methods. On one difficult task, it cut the word error rate from 180.94% to 7.07%, showing that smart retrieval can make long-form speech processing both accurate and efficient.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

kv cache compression sparse attention long-context processing