A multi-agent framework for coherent video storytelling via global optimization

Co-Director uses hierarchical parameterization and a multi-armed bandit to balance exploration and exploitation, reducing semantic drift

PaperBig Techcs.AIarXiv:2604.24842v1
Yale Song · Yiwen Song · Nick Losier · Nathan Hodson · Ye Jin · Rhyard Zhu · +10 more

Google Inc.

Research Digest··2 min read
The authors propose Co-Director, a hierarchical multi-agent framework that formalizes video storytelling as a global optimization problem. They introduce a multi-armed bandit for global creative direction and a local multimodal self-refinement loop for identity and consistency. GenAD-Bench, a 400-scenario dataset, shows significant improvements over baselines.

What they did

The authors designed Co-Director as a multi-agent system with two levels: a global orchestrator using a multi-armed bandit to explore narrative strategies, and local agent modules for video generation with self-refinement to maintain character identity and sequence coherence. They created GenAD-Bench with 400 fictional product advertising scenarios for evaluation. They compared against state-of-the-art video generation pipelines.

Key findings

  • Co-Director significantly outperformed baseline methods on GenAD-Bench across semantic coherence and visual consistency metrics.
  • The hierarchical parameterization reduced semantic drift and cascading failures common in chained agent pipelines.
  • The multi-armed bandit effectively balanced exploration of novel story directions with exploitation of successful configurations.

Why it matters

This work provides a principled optimization framework for agentic video storytelling, addressing the key challenge of maintaining coherence across generated clips. The GenAD-Bench dataset enables standardized evaluation for this emerging task.

Caveats

The evaluation is limited to advertising scenarios; generalization to other narrative forms is suggested but not extensively validated. The framework relies on underlying diffusion models, so improvements in those models could yield different results. The multi-agent architecture may have increased computational cost.

§
newspaper

Research Digest

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.