World action models jointly predict actions (intent) and future video (foresight). Standard training adds noise to both simultaneously, but this creates a mismatch: perturbed actions would produce a different visual future, yet the noised video remains tied to the original recording. CtrlWAM solves this by executing perturbed actions in a simulator and rendering the resulting video, then adding noise to that rendered video. This provides physically consistent training pairs for the model to learn from.
To further improve training, CtrlWAM introduces warped video–action noise schedules that assign different noise levels to video and actions, keeping the visual layout responsive as action predictions evolve. It also extends the action interface to multiple agents, allowing a unified model to handle both predicted and commanded futures for several agents. Driving experiments show more accurate action forecasts, closer agreement between generated video and actions, and better command following. Robotics experiments demonstrate stronger motion fidelity and controllability.
These findings are interesting because they address a fundamental misalignment in how world action models are trained, leading to more controllable and consistent predictions for autonomous driving and robotic manipulation.