What they did

The authors developed Spatially Aware World Action Model (SA-WAM), which uses a single video-diffusion backbone to jointly generate future RGB observations, depth maps, and robot actions. This extends world action models beyond their usual reliance on RGB input alone.

To feed depth through the pretrained model’s frozen variational autoencoder tokenizer, they apply a nonlinear transformation that maps unbounded depth values into the tokenizer’s bounded input range. They evaluate the model on the simulated RoboCasa and LIBERO-Plus benchmarks and in real-world experiments with a UR5 robotic arm.

Key findings

• SA-WAM achieves the best reported task performance on RoboCasa and LIBERO-Plus among the baselines considered by the authors.
• Adding depth improves future-state prediction as well as action prediction, according to the reported evaluations.
• On a real UR5 arm, SA-WAM outperforms strong comparison methods, with its largest advantages appearing in randomized environments.
• The authors find that better world-model predictions correlate with higher rollout success, linking predictive quality to downstream robot performance.

Why it matters

The work shows that explicit geometric information can be incorporated into a pretrained video world model without retraining its visual tokenizer. This offers a comparatively direct route to preserving broad video-derived priors while giving robot policies access to 3D structure that may be especially useful when scenes, objects, or viewpoints vary.

Caveats

The supplied abstract does not provide numerical effect sizes, uncertainty estimates, dataset scale, or details of the real-world task suite, so the magnitude and robustness of the reported gains cannot be assessed here. The evidence is also limited to two simulation benchmarks and one robot platform, leaving generalization to other embodiments, sensors, and longer-horizon tasks open.