Reconstructing dynamic 3D scenes from egocentric video is important for robotics and augmented reality. But everyday actions like cutting or opening a container cause topological changes, where an object's parts split or surfaces appear, which break existing methods that assume the object's shape only deforms smoothly.
To address this, the authors built TopoEgo, a benchmark of 24 real-world videos capturing such events, and developed Topo-4DGS. This method separately models how different parts of an object move rigidly after a topological change, and updates its 3D representation (a set of Gaussian primitives) to add or remove surfaces as they appear or disappear.
Topo-4DGS significantly outperforms existing state-of-the-art methods on these challenging scenes, especially just after a topological event. This means more accurate 3D reconstructions for applications like robot learning and AR that need to understand real-world object interactions.