Control objectives, not reconstruction quality, should guide representation design in world-action models

The authors show that perceptual features like DINO, when calibrated and shaped by action-loss gradients, enable effective joint policy learning and world prediction without generative video pre-training.

Top University
Haoyi Jiang · Liu Liu · Xinjiang Wang · Zhihao Sun · Zequn Chen · Sen Wang · +9 more

Huazhong University of Science and Technology · D-Robotics · Horizon Robotics · Fudan University · Xi'an Jiaotong University

Research Digest··2 min read
Through controlled comparisons, the authors demonstrate that reconstruction fidelity alone is insufficient for choosing representations in world-action models, and that perceptual features benefit from calibration for dynamics modeling.

0 and RoboDojo, including Video-VAE latents from generative models and raw DINO features.

Why this paper

From Fudan University and 4 others

In one line

ReWAM shapes representations with action objectives, achieving 93.6% success on RoboTwin 2.0 without generative video pre-training.

What we could check

  • ·No code link found
  • ·No weights link found
  • ·No dataset link found
  • ·No compute details found
  • ·No stated limitations found
  • ✓Reports numbers on named benchmarks (2 benchmarks)

Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.

§

Research Digest

Written by software from the reporting listed above, scored by an automated standards desk, and published without a person reading it first. If something here is wrong, tell the editor and it will be put right.