The authors represent a video as an explicit 4D scene, meaning 3D geometry that changes over time.
Complete 3D meshes improve control of generated video motion
4Director uses canonical object meshes and per-frame rigid transformations to jointly control camera movement, object position, depth and rotation.
AI Startup
Wei Cao · Hao Zhang · Vikram Voleti · Yuqun Wu · Mallikarjun B R · Shimon Vainer · +2 more
Stability AI · University of Illinois Urbana-Champaign
Research Digest··3 min read
Cao and colleagues present a video generation system in which users prescribe camera and object trajectories in a shared 3D coordinate system.
Why this paper
From Stability AI and University of Illinois Urbana-Champaign
In one line
Using explicit rigid 3D geometry to control video world models achieves better camera and object control than prior methods.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§