RoboSPA is a large-scale dataset and benchmark for diagnosing embodied reasoning in Vision-Language-Action (VLA) models. It focuses on two core dimensions: and Long-Horizon Procedural Planning, covering 10 task categories and 56 base tasks. Each task has five difficulty levels, yielding 280 variants, and the dataset includes 527K trajectories across multiple embodiments and scenes.
Beyond simple success rates, RoboSPA introduces diagnostic metrics for detailed evaluation. Experiments on representative VLA models reveal that current systems struggle with complex spatial relations, precise low-level execution, and memory-intensive planning. These findings highlight specific weaknesses in VLA models, making RoboSPA a challenging benchmark for advancing more capable and generalizable embodied agents.