The authors pretrain a ViT-B/16 encoder from scratch using still images and paired text.
Unified visual encoder delivers state, language, and reconstruction for world models
ALPS, a ViT-B/16 pretrained from still images and text, matches the state-probe error of a 3.5x larger model while preserving language alignment and pixel decode ability
Big Tech
Yanqing Liu · Letian Zhang · Chengzhi Liu · Zijun Wang · Jieru Mei · Hongliang Fei · +2 more
University of California, Santa Cruz · University of California, Santa Barbara · Google
Research Digest··2 min read
The authors pretrain a ViT-B/16 from scratch using language similarity, pixel reconstruction, and spatial correspondence objectives, then refine it with cross-view self-distillation.
§