The authors constructed Beyond3D from the HD-EPIC dataset, which provides 3D scene reconstructions and annotations of object movements during unscripted cooking and everyday activities.
New benchmark tests if AI can track objects after they vanish from view
Beyond3D challenges vision-language models to reason about relocated objects no longer visible in egocentric video.
Big Tech
Fangzhou Ma · Ivo Alexander Ban · Eren Homburg · Gabriele Goletto · Rémi Pautrat · Mahdi Rad · +2 more
ETH Zurich · Microsoft Spatial AI Lab · Bocconi University
Research Digest··2 min read
Ma et al.
Why this paper
From Microsoft Spatial AI Lab and 2 others
In one line
Vision-language models struggle to reason about objects moved out of sight, reaching only 42.2% on a new 9,000-question egocentric video benchmark.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§