Generative Cinematographer (GenCine) addresses the ambiguity of 2D motion controls in video generation by lifting a single image into a 3D point cloud. Artists specify a camera path and move foreground regions using local rigid 3D motion handles, which can approximate non-rigid motion through independent piecewise-rigid control. These controls are projected into guidance maps (foreground identity, foreground XYZ, background XYZ) that a pretrained video can interpret. The system is trained on real and synthetic videos paired with 3D motion controls, using a lightweight side branch and LoRA adapters on a frozen base model. Experiments demonstrate consistent camera-relative motion, improved geometric consistency under viewpoint changes, and strong controllability across diverse real-world scenes. This work is interesting because it makes intended 3D motion explicit, overcoming the inherent ambiguity of image-space trajectories.
Shared 3D controls coordinate camera and object motion in generated video
GenCine converts artist-specified camera paths and local object movements into world-space guidance for a pretrained video model.
Top University
Jiahan Zhang · Chaohao Yang · Namitha Guruprasad · Vivekjyoti Banerjee · Trong-Tung Nguyen · Alan Yuille · +1 more
Johns Hopkins University
Research Digest··3 min read
Thread:Video Diffusion Scaling
In one line
Generative Cinematographer (GenCine) lifts a single image to 3D to jointly author camera and foreground motion using local 3D motion handles.
Before you read this
An idea this paper builds onRead it as
Zhang et al. present GenCine, a system for directing camera and foreground motion together from a single reference image.
The authors lift a single image into a colored 3D point-cloud scaffold.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.