Robots that follow instructions often struggle to anticipate how the world changes as they move and grasp objects, leading to unreliable actions.
A new method trains a robot to predict future color images, depth (how far surfaces are), and motion from current observations and language commands. It learns these predictions using a separate video prediction model during training, but at run time it uses only compact internal features, so the robot still reacts instantly.
In simulated tasks, this approach completed actions with 99.3% success in one benchmark and 93.6% in another. This matters because it shows robots can learn physical foresight without slowing down, making them more practical for real-world use.