The authors built UrbanGround, a sandbox replica of Hong Kong from real 3D geospatial data, to test whether multimodal large language model (MLLM) agents can turn local street-level perception into coherent spatial action over time. They designed tasks that escalate from local scene understanding to long-range navigation and dynamic environment interaction.
They found that agents show useful abilities in visual recognition and short-range spatial reasoning, but orientation and pedestrian-aware movement are unreliable. The central failure occurs during extended exploration, where local abilities do not compose into sustained goal-directed behavior and errors accumulate without correction. This is interesting because it reveals a gap between isolated perception and practical urban agency, and UrbanGround provides a standardized testbed for studying this problem.