world action model
A world action model is a type of AI that learns to predict what will happen next in a video, along with the actions that cause those changes. Think of it as a system that watches a robot's demonstration on a tabletop and learns to forecast the next frames of video and the specific arm movements required to achieve a task. These models are trained on video and action data together, helping them understand cause and effect in physical environments.
Recent research finds that explicitly generating future video frames during inference improves how well these models generalize to new situations, even though it is computationally expensive. Innovative methods like Simple-WAM streamline this process by running just one prediction step, achieving faster speeds while outperforming both slow and fast variants. Other works show these models can replay their own past experiences to avoid forgetting old tasks, learn from human demonstrations in different environments, and even derive action knowledge from observation-only videos, highlighting their broad potential.
- Papers
- 7
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Nov 2026
7 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1World action models can learn continually without old data via self-generated replay
A world action model can reduce catastrophic forgetting without prior data by replaying its own regenerated, truncated, and physically filtered rollouts, nearly matching experience replay.
Oct 2026 - 2Cross-video human demonstration pairs improve robot world-action model transfer
Cross-video prediction from task-paired human demonstrations learns transferable world-action representations that improve robot control.
Oct 2026 - 3Predicting motion, not video, improves robot learning from human footage
Predicting 3D scene flow instead of video from pretrained models improves robot policy learning.
Nov 2026
How fast it is moving
Often studied with
flow matchingA generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.3 togetherdiffusion transformerA neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.2 togethervision-language-action modelA robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.2 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.RO
World action models can learn continually without old data via self-generated replay
A world action model can reduce catastrophic forgetting without prior data by replaying its own regenerated, truncated, and physically filtered rollouts, nearly matching experience replay.
University of Electronic Science and Technology of China, Shanghai AI Lab · Oct 2026
- Academiccs.CV
Predicting motion, not video, improves robot learning from human footage
Predicting 3D scene flow instead of video from pretrained models improves robot policy learning.
Georgia Institute of Technology · Nov 2026
- Academiccs.CV
Cross-video human demonstration pairs improve robot world-action model transfer
Cross-video prediction from task-paired human demonstrations learns transferable world-action representations that improve robot control.
NAVER AI Lab, Georgia Institute of Technology · Oct 2026
- Independentcs.CV
Robot policies generalize better when future tokens remain at inference
Latent world action models match explicit ones in distribution but lose generalization; conditioning on fully noised future tokens recovers most of the benefit.
Oct 2026
- Big Techcs.CV
Observation-only videos can directly pretrain robot action policies
NAVA-WAM directly pretrains an action policy from observation-only videos using future-video flow matching and outperforms prior world action models.
Meta AI, University of Copenhagen · Oct 2026
- Big Techcs.CV
Sensorimotor alignment predicts robot task success offline
Sensorimotor alignment, an offline metric comparing sensory and motor representation neighborhoods, predicts robot task success across embodied model families.
Rice University, NVIDIA · Oct 2026
- Chinese Techcs.RO
Robot policies encode changed instructions but still follow scene cues
Vision-language robot policies often encode changed instructions but still follow scene-associated targets because language is weakly grounded in action selection.
HKU, CUHK(SZ) · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.