Spatio-temporal video grounding normally assumes that every text query refers to something present in the paired video.
Video grounding models can ignore queries and still look plausible
Tests with irrelevant or missing text reveal that current systems can rely on recurring spatial and temporal patterns rather than query-video correspondence.
Big Tech
Eryk Kołodziejczyk · Alberto Presta · Karol Szurkowski · Michal Byra
Samsung AI Center · Institute of Fundamental Technological ResearchPolish Academy of Sciences
Research Digest··3 min read
Kołodziejczyk and colleagues tested whether spatio-temporal video grounding models genuinely use language when locating an object or event in a video.
Why this paper
From Samsung AI Center and Institute of Fundamental Technological ResearchPolish Academy of Sciences
In one line
Current spatio-temporal video grounding models are insensitive to query relevance and predict plausible outputs for irrelevant queries.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§