Exemplar2VQA is a framework that automatically generates large-scale 3D spatial question-answer pairs for training multimodal large language models (MLLMs). It uses a multi-agent system where specialized agents translate user-provided exemplar questions into executable Python code, leveraging a library of geometric utilities and simulation feedback to produce accurate spatial ground truth.
Fine-tuning Qwen2.5-VL (3B and 7B) models exclusively on synthetic indoor data from Exemplar2VQA led to significant performance improvements on indoor spatial benchmarks. Impressively, the gains transferred to outdoor and mixed-scene benchmarks without any outdoor training data, demonstrating strong sim-to-real generalization.
This is interesting because it addresses a key bottleneck: the scarcity of complex 3D spatial QA data. By bypassing LLMs' inherent spatial reasoning flaws through deterministic code execution and scaling from simple exemplar templates, it offers a scalable and versatile alternative to manual annotation or direct LLM synthesis.