This paper addresses the problem of answering questions about long videos where the same object appears across hours or days. Existing methods that rely on text descriptions often fail to distinguish between different objects that look similar or to connect observations of the same object across different events.
The authors introduce a framework called Grounded Entity Biographies (GEB) that uses visual evidence to group observations of the same physical object into a 'biography'. When a question is asked, the system retrieves the relevant biography along with the specific moments, allowing it to track an object's history. On the EgoLifeQA benchmark, GEB achieved 72.0% accuracy, outperforming the prior best result by 4.4 percentage points.