Gideoni et al.
Vision and language representations may never fully align regardless of model scale
The authors demonstrate that misalignment between modalities is an inherent property of the data, leading to predictable failures in vision-language models.
Top University
Yonatan Gideoni · Yoav Gelberg · Tim G. J. Rudner · Yarin Gal
University of Oxford · University of Toronto
Research Digest··3 min read
The authors develop a theoretical framework showing that current vision and language representations are linearly unalignable due to fundamental differences in the underlying data, not model scale.
§