Visual encoding lets language models answer questions with 16 tokens instead of thousands

OpticalRAG renders documents as page images, compressing retrieved visual tokens into a tiny prefix that preserves accuracy better than textual methods.

Academic

Senhao Liu · Yuheng Zhang · Chunyu Wei · Yueguo Chen · Xinran Zhang

Renmin University of China · ComindX · Claremont McKenna College

Research Digest·12 November 2026·3 min read

Illustration: Visual encoding lets language models answer questions with 16 tokens instead of thousands

In one line

OpticalRAG compresses retrieved documents into 16 visual tokens using rendered page images, improving accuracy while drastically reducing token usage.

Before you read this

Three ideas this paper builds on

context compactionCompressing or summarizing an LLM's past context into a smaller representation to extend memory and improve reasoning within limited token budgets.79 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.250 papers →evidence retrievalPulling specific, relevant information from a large corpus to support an answer, decision, or reasoning step.106 papers →

Read it as

Large language models need to read many documents to answer questions, but each word or piece of text (called a token) costs computation. Compressing the text too much loses important details.

OpticalRAG turns each document page into an image and uses a vision model to encode that image into a small set of visual tokens. Then, a lightweight method called query-driven distillation reduces those tokens to just 16 per query, keeping only the information relevant to the question.

On several benchmarks, OpticalRAG improved accuracy by 5.17 points over a baseline without any document retrieval, while using only 16 tokens per query. This shows that compressing documents as images can preserve more information with far fewer tokens than traditional text compression, making RAG systems cheaper and faster.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

retrieval augmented generation token efficiency context compaction long-context processing vision-language model evidence retrieval