Robots complete more two-handed tasks when AI repeatedly refines actions

Gong and colleagues report a 20.8-percentage-point improvement across four real-world tasks by involving a vision-language model throughout action generation.

Big Tech

Xicheng Gong · Wenxuan Song · Chengze Liu · Qiwei Li · Qize Yu · Jiaolong Yang · +2 more

Peking University · The Hong Kong University of Science and Technology (Guangzhou) · Microsoft Research Asia

Research Digest·3 November 2026·2 min read

Illustration: Robots complete more two-handed tasks when AI repeatedly refines actions

In one line

Vision-language models that actively participate in action denoising outperform those that only provide static conditioning.

Before you read this

An idea this paper builds on

vision-language-action modelA robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.63 papers →

Read it as

Robots translate visual input and language commands into actions. Common systems combine a vision-language model with a diffusion process that cleans noise into actions. Usually the model only sets the initial condition, without updating during formation. This hurts adaptation to new situations.

JADE makes the model an active denoiser. At each step, it reads the noisy action and provides evolving guidance. The model is also trained to predict the clean action directly from noise, tightening perception-control.

On RoboDojo, JADE achieved 12.67% success rate, best among similar models. On real two-arm tasks, it raised the average success rate by 20.8 percentage points. This shows that dynamic VLM involvement improves performance and generalization.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

joint denoising vision-language-action model flow matching action tokenization adaptive sampling schedule gated cross-attention