SUAVE is a masked diffusion transformer that treats video frames, actions, and language as discrete tokens in a shared sequence. By choosing which tokens to mask at inference, the same model can act as a world model (predict future video), a robot policy (predict actions), or a video-action model (predict both). It is trained with a single unmasking objective and can co-train on labeled robot video and unlabeled human video by excluding masked action positions from the loss.
In simulation and real-world experiments, SUAVE is competitive with dedicated world models and specialized policies on static and dynamic manipulation tasks. On a real robot, it generates subgoal images and action chunks at 2.5 actions per second on an RTX 5090 GPU. Pretraining on robot video and co-training on human video significantly improves performance and zero-shot robustness, more than doubling task scores from scratch and roughly tripling them under distribution shift.
This is interesting because it shows that a single architecture can perform multiple roles without separate modules, and can leverage abundant unlabeled video data to improve robot learning.