← All threads

Robot World Models

Learning world models from robot sensory and proprioceptive data for zero-shot generalization in manipulation tasks.

71 papers · 1 months

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

Results across this thread

65 reported results from the papers in this thread.

PaperBenchmarkMetricResultConditionsCode

The table is for subscribers. Subscribe to see every reported number side by side.

How this thread developed

  1. Oct 2026 · Renmin University of China, Zhipu AI

    Canonical gripper-frame representation achieves zero-shot cross-embodiment transfer for two-finger manipulation

    Proposes canonical gripper-frame representation enabling zero-shot cross-embodiment transfer, advancing world model generalization.

  2. 5 further papers

    Oct 2026

    Robot policies generalize better when future tokens remain at inference

    Shows that retaining future token generation at inference improves generalization in robot world action models.

  3. 2 further papers

    Oct 2026 · Huawei Heisenberg Research Center, Huawei Noah’s Ark Lab

    Looking Beyond Safe Actions Helps Robots Avoid Policy Dead Ends

    formalizes why locally safe actions can lead to dead ends and proposes a policy-relative measure to avoid them

  4. Oct 2026 · Amazon FAR, UC San Diego

    One Contact Skill Transfers Dexterous Manipulation Across Robot Hands

    Introduces GOTT, a robot-agnostic contact-acquisition skill that transfers dexterous manipulation across different robot hands for zero-shot generalization.

  5. Oct 2026 · Columbia University, Toyota Research Institute

    One Model Predicts Robot Futures and Chooses Actions

    SUAVE unifies language, video, and action tokens in one autoregressive model for robot futures prediction and action selection.

  6. 2 further papers

    Oct 2026 · POSTECH, KAIST

    Forecasting 3D scene motion improves dexterous robot manipulation

    adds a world action model that jointly represents environment and hands as 3D point trajectories, improving dexterous manipulation via pretraining on human videos.

  7. Oct 2026 · University of Illinois Urbana-Champaign, Cold Spring Harbor Laboratory

    Dependency sparsity can isolate world latents without reconstructing observations

    Proposes reconstruction-free latent recovery in world representations using dependency sparsity.

  8. 4 further papers

    Oct 2026 · FAIR at Meta, Chandar Research Lab

    Latent world models for robotics scale predictably with compute

    Introduces RoboJEPA and shows latent world model performance scales predictably with compute across multi-embodiment robotic data.

  9. Oct 2026 · NVIDIA, MIT

    Autoregressive video pretraining unlocks the value of long context for real-time robot control

    Shows that scaling the context window of a causal world-action model with autoregressive video pretraining improves real-time robot control.

  10. Oct 2026 · University of Illinois Urbana-Champaign, Carnegie Mellon University

    More Egocentric Video Improves Agents but Not Their Object Effects

    Scales egocentric video for training video world models, improving human motion reproduction but not object effects.

  11. 1 further paper

    Oct 2026 · Texas A&M University, Polara Labs Inc.

    Autonomous agent uses identifiability analysis to decide why a model plateau has been reached

    Couples an LLM proposer with an identifiability engine to autonomously diagnose plateaus in mechanistic world model learning.

  12. 3 further papers

    Oct 2026 · Chinese Academy of Sciences, University of Chinese Academy of Sciences

    Systematic study reveals what matters for world modeling in robot VLAs

    Systematic study of world modeling paradigms in robot VLAs, identifying integration strategies that matter.

  13. 1 further paper

    Oct 2026 · The Hong Kong University of Science and Technology (Guangzhou), North China Electric Power University

    Context scheduling improves long-horizon robot video prediction

    Proposes context scheduling to improve long-horizon robot video prediction using autoregressive world models.

  14. 2 further papers

    Oct 2026 · HKUST (GZ), Primatix

    Unified world action model jointly learns perception, prediction, and planning

    Introduces a unified world action model that jointly learns 3D interaction reconstruction, action-conditioned prediction, and planning.

  15. Oct 2026 · The University of Sydney, Machine Learning Research, Apple

    Planning Along Latent Manifolds Avoids False Shortcuts to Distant Goals

    Shows that Euclidean distance misleads planners in latent manifolds and proposes graph shortest paths, advancing planning in learned world model spaces.

  16. 1 further paper

    Oct 2026 · University of California, Santa Cruz, University of California, Santa Barbara

    Unified visual encoder delivers state, language, and reconstruction for world models

    Pretrains a unified ViT encoder from scratch for world models using language, reconstruction, and spatial objectives, improving state representation.

  17. 1 further paper

    Oct 2026 · The Hong Kong University of Science and Technology, National University of Singapore

    Dual Exam benchmarks agents and world models separately, exposing weak coupled performance

    Provides a framework to decouple world model quality from action policy in multimodal agents across physical and digital domains, relevant to robot world model evaluation.

  18. Oct 2026 · Hunyuan, Tencent, Peking University

    Training world models for policy improvement beats optimizing pixel accuracy

    Trains video world models directly for policy improvement rather than pixel accuracy, improving manipulation task success.

  19. 6 further papers

    Oct 2026 · RoboScience, National University of Singapore

    Joint video and 3D motion prediction reduces trajectory errors by over half

    Jointly predicts future video and 3D trajectories from a generative world model, reducing trajectory errors by over half.

  20. Oct 2026 · Queensland University of Technology, CSIRO

    Replacing language-style decoding cuts robot trajectory errors by 60%

    Replaces autoregressive token decoding with a geometry-compatible head to reduce trajectory errors in robot VLMs.

  21. 1 further paper

    Oct 2026 · Hong Kong Embodied AI Lab, CUHK

    Robots can use human videos before learning their own controls

    Learns a shared motion representation across human, robot, and simulation videos for pretraining

  22. 8 further papers

    Nov 2026 · ETH Zurich, NVIDIA

    Robots can reorient objects from pictures without tracking their pose

    Trains a robot hand to reorient objects from images without explicit pose estimation, using proprioceptive feedback.

  23. 3 further papers

    Nov 2026

    World model simulator works across 22 robot platforms, enables policy learning

    Provides a large-scale multi-platform simulator world model from diverse robot demonstrations for policy learning.

  24. Nov 2026 · Georgia Institute of Technology

    Predicting motion, not video, improves robot learning from human footage

    Shows that predicting 3D motion from human video, rather than pixels, improves robot policy learning, grounding world models in motion prediction.

  25. 3 further papers

    Nov 2026 · Huawei Technologies Ltd., University of Toronto

    Robot policy outperforms larger generative models with a quarter of the parameters

    Introduces a world-action model that predicts future visual features jointly with actions, achieving strong manipulation performance with fewer parameters than generative models.

  26. Nov 2026 · Toyota Research Institute

    Finger-mounted tactile sensors outperform wrist cameras for contact-rich robot tasks

    Directly compares tactile vs. vision-only policies on six contact-rich tasks, showing that finger-mounted tactile sensors outperform wrist cameras

  27. 1 further paper

    Nov 2026 · Michigan State University, Cisco

    Robots learn better by predicting changes rather than whole scenes

    Shows that predicting action-driven change rather than full scene reconstruction yields higher robot manipulation success rates, advancing world model design for robotics.

27 of 71 papers shown