The paper introduces WorldWeaver (W^2), a streaming video diffusion model for multi-agent interactive worlds. It augments autoregressive video generation with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. These registers are grounded with supervision signals including individual agent status, bird's-eye views, and scene text.
The model uses a Mixture-of-Transformers architecture to separate world state modeling from visual frame generation, reducing conflicts between the two objectives. Experiments on two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality over approaches that only carry forward observation history.