Latent token compression cuts AI agent token use by up to 89% and improves performance

The method, Coca-Cola, encodes tool observations as continuous vectors instead of text, preserving decision-critical information for web-search agents.

Independent

Yuetong Liu · Zhehao Zhou · Yitong Zhang · Haorui Chen · Jia Li

Research Digest·5 November 2026·2 min read

In one line

Language-model agents can replace verbose tool outputs with compact learned representations that preserve next decisions, cutting token use and runtime while improving web-search task performance.

Before you read this

Three ideas this paper builds on

latent communicationLatent communication transfers information between models via internal states like KV caches instead of explicit text.15 papers →large language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.268 papers →context compactionCompressing or summarizing an LLM's past context into a smaller representation to extend memory and improve reasoning within limited token budgets.73 papers →

Read it as

When agents use tools, they often carry entire results into later steps. The context grows, costs more to process, and may bury useful facts among irrelevant details.

Coca-Cola turns each tool result into a few latent tokens: learned numerical vectors rather than readable words. It trains the compressor to preserve what the agent needs for its next action, not to recreate every detail.

On web-search questions, it reduced token use by roughly seven-tenths to nine-tenths and runtime by about a quarter to half, while improving answers. This suggests agents can pass along decision-relevant information without carrying full tool transcripts.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

latent communication large language model context compaction token efficiency action-preserving training