StructRL builds an online RL framework for long-horizon VLA tasks. It decomposes each instruction into verifiable subtasks, orders them by dependencies, and gives intermediate rewards only when prerequisites are met, with reward size based on how quickly the subtask is completed relative to demonstrations. This converts sparse terminal feedback into dense, dependency-aware supervision.
On RoboCasa365 and LIBERO-Long, using GR00T-N1.5 and pi 0.5, StructRL outperformed the evaluated online RL baselines. For instance, with GR00T-N1.5 on RoboCasa365 it reached 49.1% success rate versus 38.6% for supervised fine-tuning and 41.5% for the strongest baseline. Ablations indicate verified subtask rewards drive most of the gain, with gating and pacing adding further improvements, and the approach also works with GRPO.
The interest is that reward design for VLA post-training can be made more informative without learning a reward model. Because the subtask is automatic and environment-based, the reward signal respects task structure instead of rewarding locally plausible but invalid progress.