The authors propose LeRF, which equips a VLM with the ability to build and use explicit reference coordinate frames for perspective taking.
Training VLMs to construct reference frames improves perspective taking.
LeRF enables selective use of entity-centered coordinate frames overlaid on images to guide viewpoint-dependent spatial reasoning.
Big Tech
Bang Xiao · Wenqi Jia · Ozgur Kara · Tiancheng Shen · Yibo Yang · Bolin Lai · +2 more
Shanghai Jiao Tong University · University of California, Merced · Amazon AGI · University of Illinois Urbana-Champaign
Research Digest··3 min read
The authors introduce LeRF, a framework that trains VLMs to dynamically decide whether to construct a reference coordinate frame, ground the relevant entity, and render a directional frame on the image.
Why this paper
From Amazon AGI and 3 others
In one line
LeRF trains VLMs to construct explicit reference frames for viewpoint-dependent reasoning, improving perspective-taking performance.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§