Simify is a training-free framework that enables robots to perform spatial reasoning at test time. It starts by reconstructing object geometries from a single RGB-D image using 3D generative models and vision-language models, creating a simulation-ready digital twin of the scene.
Given a task specified by a reward function (e.g., build the tallest tower), Simify launches thousands of parallel physics rollouts in simulation and uses an to optimize object arrangements, typically converging within seconds. Real-robot experiments show that this approach outperforms prior foundation model-based methods for spatial reasoning, highlighting the importance of complete and accurate geometry for successful sim-to-real transfer.