Robot policies that predict future camera views often waste effort on static backgrounds, because little changes between frames. Learning only the difference, what actually moves, gives the robot a clearer signal about how its actions affect the world.
The authors introduce a method called visual residual learning. A frozen vision model extracts features from the current and a future frame. The model then predicts only the feature difference, which naturally focuses on the robot arm and objects it touches.
This approach, called ResWAM, achieved top success rates on several benchmarks using far less computation than prior methods. It performed 10 percentage points better than a leading baseline on real robot tasks, with about 92.7 milliseconds latency.
