The paper introduces a framework that combines with backup-based safety filtering to handle hazards that emerge from physical interactions, like a falling ceiling fixture. The VLM predicts events and their timing, which are converted into object trajectories and occupancy sets with calibrated uncertainty via conformal prediction. This occupancy is used to constrain the robot's nominal input, ensuring a backup maneuver remains feasible.
In MuJoCo simulations with a Unitree Go1 robot, the framework achieved a 99.3% safe episode rate across scenarios involving falling fixtures, support loss, and contact propagation, compared to 43.3% for a baseline that only considers current geometry. This demonstrates the value of anticipating future changes rather than reacting to the present.