Images are normally described by continuous features; converting them to discrete tokens, like words, loses detail. This loss was assumed to make discrete representations fundamentally worse than continuous ones. The team found reconstruction error didn't predict understanding: what mattered was the bit budget, the total information the tokens carry. They introduced Factorized (FVQ), compressing complementary image views with parallel codebooks to grow that budget without saturation. FVQ matched continuous-feature models and beat a pair of strong discrete-token models by 22.8% and 59.2% on a combined benchmark. Discrete tokens can therefore serve as a scalable interface for unified .
Quantization error doesn't determine semantic fidelity of discrete visual tokens; bit budget does
A new Factorized Vector Quantization method allows discrete visual representations to match continuous ones in multimodal understanding once the bit budget is large enough.
Top University
Jialv Zou · Xiaotong Li · Zhengrong Yue · Xu Huang · Yaxin Luo · Wenhao Yan · +3 more
Huazhong University of Science and Technology · Meituan LongCat · Shanghai Jiao Tong University · Peking University · Mohamed bin Zayed University of Artificial Intelligence
Research Digest··3 min read
In one line
Discrete visual tokens with large enough bit budgets can match continuous representations in semantic fidelity.
Before you read this
Two ideas this paper builds onvector quantizationVector quantization encodes data as indices into a fixed set of codebook vectors, reducing storage and computation at some cost in precision.9 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.203 papers →
Read it as
The authors challenge the widespread assumption that quantizing continuous visual features into discrete tokens inevitably causes semantic loss.
The authors treated multimodal understanding as a proxy for semantic fidelity: a representation that supports answering any image-centric question can be regarded as semantically lossless with respect to the original visual signal.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.