SciHorizon-eLab is a system that automatically compiles natural-language scientific protocols into executable, certified embodied tasks in simulation. It handles semantic grounding, executable program synthesis, and multi-stage simulation-based certification to ensure consistency before a task is included in a .
Using this pipeline, the authors constructed SciVLABench, a benchmark of 300 certified tasks derived from 51 source scenarios. Evaluation across representative tasks revealed the strongest policy achieved only a 49.7% success rate, with particularly pronounced difficulties in human-agent coordination.
The work is significant because it automates the traditionally manual process of constructing simulation benchmarks, enabling the development and evaluation of scientific embodied agents at greater scale and reproducibility.