The authors designed a benchmark where LLMs interact with physical environments through a harness system that issues commands to devices and receives feedback.
LLMs struggle to turn words into reliable physical actions
A benchmark of 11 interactive tasks shows persistent failures in spatial awareness, error recovery, and long-term planning
Big Tech
Xinnan Dai · Kaiqi Yang · Yaxin Li · Cheng Luo · Wanli Li · Mingqing Xiao · +6 more
Michigan State University · Zhejiang University · Microsoft · University of Rochester · University of Wisconsin–Madison
Research Digest··2 min read
The authors introduce Words to World (W2W-Bench), a benchmark that places large language models in closed-loop control of real robotic arms, cameras, and sensors.
Why this paper
From Microsoft and 4 others
In one line
Large language models fail at closed-loop physical interaction due to spatial, temporal, and recovery errors.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§