Reconstructing a moving 3D scene from a video is hard because camera motion and object deformation are tangled, and the math involved often gets stuck in poor solutions. XM4D breaks this into two simpler stages: first estimate the camera movement using static parts of the scene, then estimate the object deformation separately. By turning each stage into a special optimization that guarantees a globally best solution, XM4D achieves accurate camera poses and state-of-the-art novel-view rendering on multiple benchmarks, all while scaling to thousands of frames.
Video reconstruction can certify optimal estimates of camera and scene motion
The authors separate camera tracking from scene deformation, making both geometric problems amenable to verifiable optimality certificates before refining the rendered images.
Independent
Haoyu Han · Jack Benarroch Jedlicki · Jiahui Lei · Heng Yang
Research Digest··3 min read
In one line
XM4D separates camera motion from non-rigid deformation, formulating each as a certifiably optimal rotation-only optimization.
Read it as
Han and colleagues present a system for reconstructing a scene’s changing three-dimensional shape from video while estimating how the camera moves.
The authors divide reconstruction into a static stage and a dynamic stage.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.