agentic video generation
Agentic video generation means an AI system that doesn't just produce a clip in one pass. Instead, it acts like a small production crew: it plans shots, calls external tools to generate footage, inspects what came out, and revises based on its own critique. This loop of construction, inspection, and revision lets the system handle long stories, keep characters consistent, and follow user instructions more reliably than a single text-to-video model.
The papers share a common thread: adding memory, planning, and self-criticism makes generated videos more coherent and consistent across shots. Several show that training the agent through reinforcement learning, not just inference-time fixes, improves performance. Others warn that judge models can be fooled by auxiliary text, and that strong general ability doesn't guarantee good visual results.
- Papers
- 21
- Released code
- 2
- First seen
- Apr 2026
- Latest
- Nov 2026
18 papers in the last two months, against 1 in the two before.
Start here
Three papers to read first, in the order they came out- 1AI model controls characters in unfamiliar game worlds in real time
An agentic game world model that generalizes zero-shot to novel scenes, characters, and motions with real-time interaction.
Oct 2026 - 2AI video generation gains emotional fidelity when rewards trust only reliable cues
EmoReward adaptively calibrates reward weights in emotional video generation using reliability learned from previous training run traces.
Nov 2026 - 3Video generators can better follow physics with code-built scene drafts
Video-Coder improves video generation by having an AI agent write and execute code to build a 3D scene before rendering the final video.
Nov 2026
How fast it is moving
Often studied with
world model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.10 togethervision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.7 togetherverificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.4 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.CV
AI video generation gains emotional fidelity when rewards trust only reliable cues
EmoReward adaptively calibrates reward weights in emotional video generation using reliability learned from previous training run traces.
The Hong Kong University of Science and Technology, Tencent Hunyuan · Nov 2026
- Big Techcs.CV
Video generators can better follow physics with code-built scene drafts
Video-Coder improves video generation by having an AI agent write and execute code to build a 3D scene before rendering the final video.
University of Central Florida, University of Würzburg · Nov 2026
- Top Universitycs.CV
Streaming video generators can now output frames at near-watchable speed
Streaming video generation can emit one finished chunk per concurrent forward round by pipelining denoising stages across model replicas, preserving quality while speeding throughput.
Shanghai Jiao Tong University, Xi’an Jiaotong University · Nov 2026
- Chinese Techcs.CV
AI model controls characters in unfamiliar game worlds in real time
An agentic game world model that generalizes zero-shot to novel scenes, characters, and motions with real-time interaction.
Tencent, National University of Singapore · Oct 2026
- Chinese Techcs.CV
AI video generators can follow new commands while remembering earlier actions
Hy-Nomas is a video world model that generates interactive visual worlds from free-form text and camera instructions across multiple turns.
Tencent Hy Visual Generation Team · Oct 2026
- Research Labcs.CR
Execution Logs Can Mislead Visual Judges in Video-Generation Agents
Execution traces cause video-generation verifiers to judge clips based on text, not visuals.
RIKEN · Sept 2026
- Chinese Techcs.CV
Self-evolving AI agent generates cinematic multi-shot videos by iteratively directing and critiquing itself
A unified Director Critic vision-language policy trained with alternating reinforcement learning improves cinematic multi-shot video generation.
Beijing University of Posts and Telecommunications, Peking University · Oct 2026
- Big Techcs.CV
Generating B-roll footage grounded in a user's captured video collection.
MemComposer generates B-roll sequences from user video collections that are grounded in the visual world and adhere to natural language directives.
Adobe Research, Czech Technical University · Oct 2026
- Chinese Techcs.CV
Tracking world states improves coherence in generated video stories
StoryEngine uses a state-grounded agentic framework to produce coherent and consistent long-form video stories.
Tencent, National University of Singapore · Sept 2026
- Top Universitycs.CV
Reinforcement learning trains video AI agents to use external tools effectively
A multimodal agent trained via multitask reinforcement learning coordinates external tools to improve video generation quality and accuracy.
Princeton University, Stanford University · Sept 2026
- Top Universitycs.CVcode
Multi-agent framework coordinates narrative and visual consistency for long-form video
ViMax uses coordinated multi-agent collaboration to generate long-form videos with consistent narrative and visual coherence across scenes.
The University of Hong Kong, South China University of Technology · June 2026
- Big Techcs.CV
More Egocentric Video Improves Agents but Not Their Object Effects
Scaling ego-centric video to 30,000 hours models agents well but leaves object interaction fidelity far behind, requiring training changes, not just more data.
University of Illinois Urbana-Champaign, Carnegie Mellon University · Oct 2026
- Big Techcs.CV
Evolving physics captions improve video models’ physical plausibility
Using a self-evolving language framework improves video world model physical plausibility without extra visual signals.
NVIDIA, University of Oxford · Oct 2026
- Big Techcs.CV
Video models reason better when trained to anticipate future states
ProAR makes autoregressive video reasoning more goal-directed by predicting outcomes and aligning current representations with future transitions, improving accuracy and training efficiency.
The Hong Kong Polytechnic University, University of California, Davis · Oct 2026
- Chinese Techcs.CVcode
Stateful code revision improves programmable image and video generation
MaLiang-Harness bridges the program-to-visual gap by enabling MLLMs to iteratively construct, inspect, and revise executable visual programs for image and video generation.
National University of Singapore, Fudan University · Sept 2026
- Chinese Techcs.CV
Language enables precise character and camera control in video worlds
Language instructions can precisely control video generation by reusing the model's existing semantics with minimal adaptation.
Tencent, National University of Singapore · Sept 2026
- Big Techcs.AI
A multi-agent framework for coherent video storytelling via global optimization
Co-Director formalizes video storytelling as a global optimization using hierarchical multi-agent framework with multi-armed bandit and self-refinement loops.
Google Inc. · Apr 2026
- Big Techcs.CV
Video generators can nearly halve history storage while improving reconstruction quality
A compression scheme for cached attention state in autoregressive video models preserves generation quality better by tuning Keys to match downstream outputs rather than to reconstruct the raw cache.
Zhejiang University, NVIDIA · Nov 2026
- Chinese Techcs.CV
Video generators follow user-defined rules better when separate software controls object behavior
BoxWorld separates interaction reasoning from generative rendering to create real-time interactive video worlds with user-defined behaviors.
University of British Columbia, Vector Institute for AI · Oct 2026
- Chinese Techcs.CV
AI game worlds can support multiplayer without additional training
PlayGen2 separates learned 3D world-state evolution from neural rendering to enable real-time, persistent, interactive game worlds.
The Chinese University of Hong Kong, Wellington College · Nov 2026
- Chinese Techcs.CV
Executable code gives generative worlds persistent rules and evolving state
CoDeR combines explicit code-based world rules with diffusion video generation to enable open-ended world simulation.
USTC, TongYi Lab · Sept 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.