The authors introduce a synthetic shape-counting task where each instance can be specified equivalently as an image or as text (line coordinates).
Vision-language alignment fails to improve text-only spatial reasoning
Standard VQA training does not make language models build visual representations from text, but self-distillation can induce this capability.
Academic
Shashwat Singh · He He · Tal Linzen
New York University
Research Digest··2 min read
Singh et al.
§