Small vision-language AI models can be compressed fourfold with only 1% accuracy loss

A progressive quantization training method restores multimodal understanding before language modeling and uses self-relabeled data to achieve near lossless compression for models up to 8B parameters.

Top University

Yufei Xue · Yushi Huang · Jiawei Shao · Pingcheng Dong · Yonghao Tan · Shiyao Li · +2 more

The Hong Kong University of Science and Technology · Northwestern Polytechnical University · China Telecom (TeleAI) · Tsinghua University

Research Digest·6 November 2026·3 min read

Illustration: Small vision-language AI models can be compressed fourfold with only 1% accuracy loss

In one line

ProQuant makes low-bit W4A8 quantization of small multimodal LLMs nearly lossless by progressively restoring multimodal understanding before language modeling.

Before you read this

Three ideas this paper builds on

quantization-aware trainingTraining a neural network with simulated quantization in the forward pass so its weights and activations adapt to low-precision formats, improving deployment accuracy.20 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.240 papers →post-training quantizationQuantizing pre-trained neural network weights and activations to47 papers →

Read it as

Multimodal AI models understand images and text but are too large for phones and edge devices. Quantization shrinks the model by reducing number precision, but at very low precision accuracy drops sharply, especially for smaller models best suited to devices.

ProQuant is a training method that recovers this accuracy. It re-labels fine-tuning data to match what the model already knows, and restores abilities in stages: first image understanding, then language skills.

On a small edge-ready model, ProQuant improved accuracy by roughly 6.6 percentage points over existing compressed models at very low precision. Output stayed within about 1 percentage point of the full-precision original, so edge devices can run capable multimodal AI with almost no quality loss.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

quantization-aware training progressive block activation vision-language model self-relabeling post-training quantization vision-sensitivity factor