Physical AI, intelligence for physical tasks like robotics and driving, needs realistic virtual worlds that react instantly to actions. Creating these is a major challenge.
The framework adapts Cosmos, a pre-trained world model that understands physical dynamics, to generate visual scenes in real time based on commands or agent actions. It can predict the visual result of an action or render a scene from a simulator's physical state.
This approach works across camera navigation, robot manipulation, controlled video generation, and multi-view autonomous vehicle simulation, providing a shared foundation for building interactive training environments essential for safer and more capable Physical AI.