, a script and a few visual references) and factor-aware control over character, scene, action, and camera.
Three-stage training makes MLLM latents effective for cinematic video generation
Factor-Aware Predictive Learning uses latent grounding, factor-span replacement, and fixed-generator optimization to bridge multimodal understanding and controllable generation.
Chinese Tech
Songlin Yang · Zhe Wang · Xiaotong Zhao · Xi Wang · Ran Zhang · Shuai Li · +5 more
The Hong Kong University of Science and Technology · Tencent Video · École Polytechnique
Research Digest··3 min read
The authors propose Factor-Aware Predictive Learning (FAPL), a three-stage training procedure that makes the representations of a frozen multimodal large language model directly actionable for a Video Diffusion Transformer in cinematic generation.
§