Robot World Models
Learning world models from robot sensory and proprioceptive data for zero-shot generalization in manipulation tasks.
71 papers · 1 months
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
65 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Oct 2026 · Renmin University of China, Zhipu AI
Canonical gripper-frame representation achieves zero-shot cross-embodiment transfer for two-finger manipulation
Proposes canonical gripper-frame representation enabling zero-shot cross-embodiment transfer, advancing world model generalization.
5 further papers
Oct 2026
Robot policies generalize better when future tokens remain at inference
Shows that retaining future token generation at inference improves generalization in robot world action models.
2 further papers
Oct 2026 · Huawei Heisenberg Research Center, Huawei Noah’s Ark Lab
Looking Beyond Safe Actions Helps Robots Avoid Policy Dead Ends
formalizes why locally safe actions can lead to dead ends and proposes a policy-relative measure to avoid them
Oct 2026 · Amazon FAR, UC San Diego
One Contact Skill Transfers Dexterous Manipulation Across Robot Hands
Introduces GOTT, a robot-agnostic contact-acquisition skill that transfers dexterous manipulation across different robot hands for zero-shot generalization.
Oct 2026 · Columbia University, Toyota Research Institute
One Model Predicts Robot Futures and Chooses Actions
SUAVE unifies language, video, and action tokens in one autoregressive model for robot futures prediction and action selection.
2 further papers
Oct 2026 · POSTECH, KAIST
Forecasting 3D scene motion improves dexterous robot manipulation
adds a world action model that jointly represents environment and hands as 3D point trajectories, improving dexterous manipulation via pretraining on human videos.
Oct 2026 · University of Illinois Urbana-Champaign, Cold Spring Harbor Laboratory
Dependency sparsity can isolate world latents without reconstructing observations
Proposes reconstruction-free latent recovery in world representations using dependency sparsity.
4 further papers
Oct 2026 · FAIR at Meta, Chandar Research Lab
Latent world models for robotics scale predictably with compute
Introduces RoboJEPA and shows latent world model performance scales predictably with compute across multi-embodiment robotic data.
Oct 2026 · NVIDIA, MIT
Autoregressive video pretraining unlocks the value of long context for real-time robot control
Shows that scaling the context window of a causal world-action model with autoregressive video pretraining improves real-time robot control.
Oct 2026 · University of Illinois Urbana-Champaign, Carnegie Mellon University
More Egocentric Video Improves Agents but Not Their Object Effects
Scales egocentric video for training video world models, improving human motion reproduction but not object effects.
1 further paper
Oct 2026 · Texas A&M University, Polara Labs Inc.
Autonomous agent uses identifiability analysis to decide why a model plateau has been reached
Couples an LLM proposer with an identifiability engine to autonomously diagnose plateaus in mechanistic world model learning.
3 further papers
Oct 2026 · Chinese Academy of Sciences, University of Chinese Academy of Sciences
Systematic study reveals what matters for world modeling in robot VLAs
Systematic study of world modeling paradigms in robot VLAs, identifying integration strategies that matter.
1 further paper
Oct 2026 · The Hong Kong University of Science and Technology (Guangzhou), North China Electric Power University
Context scheduling improves long-horizon robot video prediction
Proposes context scheduling to improve long-horizon robot video prediction using autoregressive world models.
2 further papers
Oct 2026 · HKUST (GZ), Primatix
Unified world action model jointly learns perception, prediction, and planning
Introduces a unified world action model that jointly learns 3D interaction reconstruction, action-conditioned prediction, and planning.
Oct 2026 · The University of Sydney, Machine Learning Research, Apple
Planning Along Latent Manifolds Avoids False Shortcuts to Distant Goals
Shows that Euclidean distance misleads planners in latent manifolds and proposes graph shortest paths, advancing planning in learned world model spaces.
1 further paper
Oct 2026 · University of California, Santa Cruz, University of California, Santa Barbara
Unified visual encoder delivers state, language, and reconstruction for world models
Pretrains a unified ViT encoder from scratch for world models using language, reconstruction, and spatial objectives, improving state representation.
1 further paper
Oct 2026 · The Hong Kong University of Science and Technology, National University of Singapore
Dual Exam benchmarks agents and world models separately, exposing weak coupled performance
Provides a framework to decouple world model quality from action policy in multimodal agents across physical and digital domains, relevant to robot world model evaluation.
Oct 2026 · Hunyuan, Tencent, Peking University
Training world models for policy improvement beats optimizing pixel accuracy
Trains video world models directly for policy improvement rather than pixel accuracy, improving manipulation task success.
6 further papers
Oct 2026 · RoboScience, National University of Singapore
Joint video and 3D motion prediction reduces trajectory errors by over half
Jointly predicts future video and 3D trajectories from a generative world model, reducing trajectory errors by over half.
Oct 2026 · Queensland University of Technology, CSIRO
Replacing language-style decoding cuts robot trajectory errors by 60%
Replaces autoregressive token decoding with a geometry-compatible head to reduce trajectory errors in robot VLMs.
1 further paper
Oct 2026 · Hong Kong Embodied AI Lab, CUHK
Robots can use human videos before learning their own controls
Learns a shared motion representation across human, robot, and simulation videos for pretraining
8 further papers
Nov 2026 · ETH Zurich, NVIDIA
Robots can reorient objects from pictures without tracking their pose
Trains a robot hand to reorient objects from images without explicit pose estimation, using proprioceptive feedback.
3 further papers
Nov 2026
World model simulator works across 22 robot platforms, enables policy learning
Provides a large-scale multi-platform simulator world model from diverse robot demonstrations for policy learning.
Nov 2026 · Georgia Institute of Technology
Predicting motion, not video, improves robot learning from human footage
Shows that predicting 3D motion from human video, rather than pixels, improves robot policy learning, grounding world models in motion prediction.
3 further papers
Nov 2026 · Huawei Technologies Ltd., University of Toronto
Robot policy outperforms larger generative models with a quarter of the parameters
Introduces a world-action model that predicts future visual features jointly with actions, achieving strong manipulation performance with fewer parameters than generative models.
Nov 2026 · Toyota Research Institute
Finger-mounted tactile sensors outperform wrist cameras for contact-rich robot tasks
Directly compares tactile vs. vision-only policies on six contact-rich tasks, showing that finger-mounted tactile sensors outperform wrist cameras
1 further paper
Nov 2026 · Michigan State University, Cisco
Robots learn better by predicting changes rather than whole scenes
Shows that predicting action-driven change rather than full scene reconstruction yields higher robot manipulation success rates, advancing world model design for robotics.
27 of 71 papers shown