The authors compared matched models trained with either image patch embeddings or content-free padding tokens, varying model size and training compute.
Vision scales multimodal learning without altering pure-text loss
Controlled ablations show visual content improves multimodal text prediction as a power law of compute while preserving language benchmarks and enabling selective spatial transfer.
Chinese Tech
Haoyuan Wu · Aoqi Wu · Jiani Hu · Bei Yu
The Chinese University of Hong Kong · Tencent
Research Digest··3 min read
The authors trained native multimodal models with either real image embeddings or content-free padding, holding text supervision and training configuration constant across model sizes and compute budgets.
Why this paper
From Tencent and The Chinese University of Hong Kong
In one line
Visual content in native multimodal pre-training improves text prediction on image-text data and spatial reasoning via text, scaling with compute while preserving language performance.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§