Cached reasoning traces match frontier LLMs at a fraction of the cost

The authors show that once a language model has solved a class of problems, its reasoning can be distilled into a CPU-executable cache that performs on par with the original model across difficult math and science tasks.

Independent

Peter Belcak · Greg Heinrich · Saurav Muralidharan · Shizhe Diao · Pavlo Molchanov

Research Digest·9 November 2026·2 min read

Illustration: Cached reasoning traces match frontier LLMs at a fraction of the cost

In one line

Agentic reasoning caching converts LLM reasoning traces into CPU-executable caches that perform on par with frontier LLMs at drastically lower cost.

Before you read this

An idea this paper builds on

kv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.73 papers →

Read it as

Large language models solve every problem from scratch, even very similar ones. This is slow and expensive.

The cache stores the reasoning from an LLM solution and reuses it on a CPU for similar problems. This is called 'agentic reasoning caching'.

These cached solutions match the performance of best LLMs on hard math and science tasks. They are fully explainable, can be updated, and cost between 16 and 291 times less to run.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

agentic reasoning caching system-level caching cost-quality trade-off in-context learning chain of thought kv cache compression