Creating interactive video worlds where users can act and see consequences in real time is hard. Current methods try to learn everything inside a video generator, making it difficult to specify object-level rules and control outcomes. BoxWorld separates the problem: a Box World Engine applies user-specified rules to actions and tracks object states, while a Box-State Generative Renderer turns that state stream into video. This allows fine-grained control, like a door opening only after a cat collects a key. BoxWorld substantially outperforms end-to-end world models on a new benchmark for interaction control while running in real time.
Separating rule-based reasoning from video generation enables controllable interactive worlds
BoxWorld uses a lightweight state interface and a three-stage training strategy to let users specify object-level behaviors in real-time generated video.
Chinese Tech
Zike Wu · Ahmed Hatem · Chintan Trivedi · Wallace Michel Pinto Lira · Xuebin Qin · Lele Wang · +1 more
University of British Columbia · Vector Institute for AI · Huawei Canada · Canada CIFAR AI Chair
Research Digest··3 min read

In one line
BoxWorld separates interaction reasoning from generative rendering to create real-time interactive video worlds with user-defined behaviors.
Before you read this
Three ideas this paper builds onrole decouplingSeparating distinct functions such as planning, reasoning, and execution into separate components improves efficiency, quality, and reliability.39 papers →structured layout reasoningA multi-step process that generates or verifies spatial arrangements of objects against explicit constraints and geometry.14 papers →coding agentAn AI-powered software system that autonomously writes, modifies, or extends source code based on high-level instructions.130 papers →
Read it as
The authors propose BoxWorld, a framework for real-time interactive video worlds where user-defined rules control object behaviors.
The authors introduce BoxWorld, a framework that decouples interaction reasoning from generative rendering for interactive video worlds.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.