Visual lookup memory scales VLM capacity without increasing per-token compute

The authors propose V-LUMEN, which discretizes visual features and uses spatial and text-conditioned hashing to retrieve embeddings from a large table, improving a frozen VLM on reasoning benchmarks.

Big Tech

Miso Choi · Daekeun Kim · Eunji Kim · Jungbeom Lee

Korea University · Amazon

Research Digest·28 October 2026·2 min read

Illustration: Visual lookup memory scales VLM capacity without increasing per-token compute

In one line

V-LUMEN adds external lookup memory to vision-language models, discretizing visual features so capacity grows in storage, not per-token compute.

Before you read this

An idea this paper builds on

vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.203 papers →

Read it as

(VLMs) answer questions about images. Adding capability usually means adding parameters, which costs more computing per image. V-LUMEN adds a separate lookup table, like a dictionary of visual facts, that the model queries.

For each image patch, the model assigns a code from a fixed dictionary, then uses a hash (a fast way to find a slot) based on that code, the patch's location, and the question, to fetch relevant entries. The fetched entries are merged into the model's processing while the model's existing weights stay frozen (unchanged).

On five reasoning benchmarks, this improved a model with 2 billion parameters, and transferred to new image types without retraining. It matched the accuracy of fine-tuned adapters (small extra modules) while using less per-image compute, because the extra knowledge lives in storage, not in computation.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

vision-language model visual lookup memory embedding scaling spatial aggregation-based hashing text-conditioned hashing non-parametric addressing