Chain-of-Dynamics reasoning with world imagination enables robot learning from human videos without action labels

ThinkWorld uses a sequence of discrete motion tokens to represent actions across embodiments and reinforces them via future state prediction, achieving human-to-robot transfer.

Big Tech

Chaehyun Kim · Chi-Pin Huang · Jisoo Kim · Min-Hung Chen · Yu-Chiang Frank Wang · Seungryong Kim · +1 more

NVIDIA · KAIST

Research Digest·28 October 2026·2 min read

Illustration: Chain-of-Dynamics reasoning with world imagination enables robot learning from human videos without action labels

In one line

Representing actions as chains of discrete motion tokens that are reinforced by how well they predict future video frames lets unlabeled human videos train robot manipulation policies.

Before you read this

Three ideas this paper builds on

world model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.90 papers →vision-language-action modelA robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.56 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.223 papers →

Read it as

Robot demonstrations that include the robot's own actions are expensive to collect, while human videos are plentiful but lack those actions. ThinkWorld aims to learn from human video without losing the precise grounding that labeled demonstrations provide.

It represents each action as a Chain-of-Dynamics: a sequence of discrete motion tokens describing how the world changes from one state to the next. The model learns this format from labeled robot demonstrations, then for any video imagines the future the chain would cause and rewards chains that match the observed outcome.

Tests in simulation and on real robots show human-to-robot transfer and strong generalization to unseen tasks and environments, suggesting robot learning can scale with abundant human video.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

chain of thought world model for agents vision-language-action model reinforcement learning embodied artificial intelligence