ui grounding
Mapping natural language commands or descriptions to specific visual elements of a user interface.
- Papers
- 7
- Released code
- 0
- First seen
- Aug 2026
- Latest
- Sept 2026
7 papers in the last two months, against 0 in the two before.
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.CL
Hybrid agents learn to recreate software across five computing platforms
Alibaba Token Hub, Alibaba Group · Sept 2026
- Independentcs.AI
Training agents to combine graphical interfaces and commands improves efficiency
Sept 2026
- Big Techcs.CV
A single visual agent spans mobile, desktop, web, and tool use
Apple · Sept 2026
- Top Universitycs.AI
Markdown interfaces cut agent context costs without reducing task success
Seoul National University, H1R.AI · Sept 2026
- Industrycs.AI
Decouple high-level VLM planning from low-level execution to speed up mobile agents
Rice University · Sept 2026
- Top Universitycs.AI
Cross-device workflows expose major weaknesses in today’s GUI agents
Beihang University, Beijing Institute of Technology · Sept 2026
- Top Universitycs.AI
Structured software interfaces outperform screenshot-and-click control for AI agents
X-LANCE Lab, Shanghai Jiao Tong University · Aug 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.