← All concepts

prefix caching

Papers
4
Released code
0
First seen
Oct 2026
Latest
Nov 2026

4 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Simple LRU rivals sophisticated eviction for LLM prefix caching

    Prefix cache management should prioritize recency and selectively add quick demotion, compute-aware eviction, and capacity-dependent granularity.

    Oct 2026
  2. 2Chunked prefill, not cache architecture, drives LLM serving engine speed differences

    Most of a large benchmarked gap between two LLM serving engines comes from one scheduler default, chunked prefill, not from cache design.

    Nov 2026
  3. 3AI agent servers can stay overloaded after traffic bursts end

    A transient burst can permanently destroy prefix-cache hit rates and throughput in LLM agent serving.

    Nov 2026

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.